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Executive summary 


In 2017, the government launched Career Learning Pilots in order to test the design and 
provision of various training approaches for working adults with low to intermediate skills. 
As part of this initiative, the Department for Education provided approximately £11.3 
million in funding to pilot projects that offered flexible and accessible ways of delivering 
learning to adults with low or intermediate skills who were either in work or seeking to 
return to the workplace. Insights from these pilots will be used to inform adult skills policy, 
including the National Skills Fund, which, starting this Parliament, will provide £2.5 billion 
to help adults learn valuable skills and prepare for the economy of the future. 


Eligible projects either designed or enhanced a method of delivering learning that was 
not currently widely available and that was accessible to in-work adults or labour market 
returners. 


Thirty-one projects were awarded funding based on their ability to meet the required 
eligibility criteria. One of the projects subsequently withdrew from the pilot, leaving 30 
projects. Following a development period in 2017/18 all projects were required to trial 
delivery with learners during the 2018/19 academic year. 


Kantar’s Public Division was appointed as the external evaluator for the FLF in 2018. 
Evaluation approach: The evaluation had four key aims: 
1. To measure learner uptake and attainment compared with standard modes of 
delivery. 


2. To identify what proportion of adults enrolled on the course were the target group. 


3. To evaluate the implementation of the FLF to provide formative feedback on 
processes. 


4. To identify the scalability of different aspects of the FLF pilots. 


The evaluation took a mixed-method approach, using a mixed-mode survey of FLF 
learners and analysis of administrative data from the ILR. It also included qualitative 
research involving in-depth case studies with eleven pilot providers, in-depth telephone 
interviews with a representative from 15 other lead providers not included in the case 
studies, and telephone interviews with 40 learners around six months after their course 
had finished. 


Evaluation key findings: 


Learner profile and course uptake: 


According to the ILR data more than half of learners were employed and just over 
a third had achieved the equivalent of a level 2 qualification or below prior to 
starting their course and were therefore members of the target audience. 


Slightly more than half of learners surveyed reported some form of caring 
responsibility showing that most learners were likely to be fitting study around 
other responsibilities, potentially benefitting from flexible learning options. 


Recruitment of learners to the FLF was more challenging than expected for most 
projects. 


Four projects clearly exceeded their number of target learners, and a further four 
projects were roughly in line with their target. However, for 21 projects, uptake was 
below the target set. 


Overall, 5,362 adult learners participated in FLF courses against a combined 
target of 26,475,' which was 20% of the initial total target showing that providers 
found it hard to estimate likely uptake and recruitment of learners was often more 
challenging than expected. 


It is important to recognise that some courses within projects were not covered in 
this analysis. It is possible that some of these were successful in widening access 
for these additional courses. As such, these findings may not be applicable to all 
flexible courses. 


The evaluation also compared uptake among FLF supported? courses with uptake for 
similar courses which were not supported by the FLF. This aimed to: 


A 


2. 


Understand whether 2018/19 uptake on FLF-supported courses was higher than 
expected based on uptake for similar courses which were not supported by the 
FLF. 


Understand to what extent such an increase in uptake could be plausibly explained 
by the FLF. 


1 This was calculated based on the estimated number of learners each learning provider told Kantar they 
expected to recruit. 


? An FLF course is one at least partly funded by the Flexible Learning Fund. 
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For eight of FLF projects uptake was larger than expected and it is likely that the FLF 
contributed to this increase in uptake. There were ten other FLF projects where uptake in 
these courses in 2018/19 was higher than expected based on the comparison courses. 
However, in these cases it was less clear that the increase in uptake was associated with 
the FLF. This demonstrates that these kinds of programmes can be effective at 
increasing uptake. 


Designing and adapting the course: 


e Courses were better designed where course objectives were effectively and 
efficiently adapted to reflect the mode of the course delivery, the needs of the 
target audience and the intended outcome of the course. 


e The key approaches to adapting course content included streamlining, tailoring 
and collating content. 


e All three approaches benefitted from partnerships with employers who could 
provide subject-specific expertise and where there they could draw on existing 
relationships. 


e Providers also reported that course adaptation was improved by an iterative 
approach. Although this was resources intensive, it resulted in more refined 
content for learners that generally received positive feedback. 


Streamlining material was prioritised when providers moved courses online, if they 
already had substantial resources, or if they had a more detailed knowledge of the 
existing content. In these cases, they identified the core topics to cover, but made sure 
that the content was aligned with qualification or partner needs. An alternative approach 
was tailoring the content, often for a specific partner employer or organisation who could 
work with the provider to identify skills gaps that a course could fill. This improved 
relationships with employers. Some providers working in a consortium adapted content 
by collating their resources with partner organisations. They aimed to identify the highest 
quality resources and combine them. This was time consuming, but providers thought it 
was beneficial. 


Where course material needed to be digitised, communication and time management 
were key to ensure this was done effectively. Where providers worked with an in-house 
developer both partiers noted that working within the same organisation gave them an 
understanding of each other’s needs. When providers were new to developing online 
materials, they suggested that communication needed to be very clear, establishing core 
expectations from the beginning. 
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Learner experience: 


Career progression was a key motivating factor for many learners. Younger 
learners were more ambitious for career advancement while older learners were 
more likely to say they wanted to improve in the job they had. 


Around half of learners surveyed said that their course was delivered in-person 
only, just over a quarter said their course was delivered through a mix of online 
and “in person” teaching, and just over a fifth said their course was delivered 
online only. 


More than half of ‘in person’ learners said that at least some of the teaching was 
delivered outside of a school or FE setting. 


Online learners found the online delivery helpful. More than half of these learners 
said that they would not have been able to do the same course if it were delivered 
entirely in person during the day. Evening courses would also have been difficult 
for learners who took part in courses taught exclusively online. 


Around a fifth had chosen a flexible course to fit around their work schedule, this 
rose to 44% of those whose course was completely online. One in ten learners 
had chosen a flexible course to fit around childcare responsibilities. 


Learners were most likely to drop out of their course because it was difficult to 
juggle learning with work, childcare or other caring commitments. 


Recruitment and retention: 


Generally, providers used a multi-pronged approach to recruit learners to their 
flexible courses. According to the learner survey, the largest group of learners 
were recruited through their current or previous employer (30%). Providers also 
used other partners to recruit including colleges, charities, Jobcentre Plus and 
local government. 


Qualitative interviews demonstrated that recruitment via partnerships was more 
successful where the partner organisation had an in-depth knowledge of the target 
audience, where they understood the benefits of flexible learning to their own 
organisation and where the relationship was more mature. 


Providers recruited via word of mouth, with friends and family being a key way in 
which learners first heard about flexible courses (18%). These learners tended to 
already be considering taking up a course. 


Providers also referred those making direct enquiries towards flexible courses 
where appropriate. 11% of learners found out about courses directly through the 
learning provider. Qualitative interviews found that for this method to be effective, 
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tutors and administrative staff talking to learners needed to buy into the benefits of 
flexible learning and have the relevant knowledge to refer appropriate learners. 


As well as encouraging learners to enrol on courses, providers also used a range 
of strategies to continue learner engagement during the course and reduce drop- 
out rates. Qualitative interviews revealed that retention of learners was more 
successful where providers held a course induction at the outset of the learning 
experience, and where providers continued to engage with learners in a 
personalised way throughout the course. 


Continued engagement has implications for tutor workload. Additional tutors would 
be needed to support flexible courses should they expand. 


Course delivery: 


Overall, just over one third (37%) of learners said that the course was better than 
expected with a further 51% saying that it was as good as expected. This was 
mirrored in qualitative learner interviews. Several factors were identified as 
determining learner satisfaction. Firstly, the quality of course content was 
considered. Learners were more satisfied where content was tailored to real-world 
scenarios, where the volume and complexity allowed them to work around other 
commitments such as caring responsibilities and where supporting materials 
looked professionally designed. 


Secondly, learners considered the quality of teaching. Learners were more 
satisfied with teaching where it was personalised, for example through providing 
additional support on areas they found more difficult, where tutors were 
responsive and where their experiences were consistent with what had been ‘sold’ 
during the recruitment phase. 

Finally, learners considered the level of accessibility when thinking about course 
quality. In determining how accessible courses were, learners thought about the 
timing, location workload and the extent to which content was user-friendly. 


Course outcomes: 


Leaners generally had positive views on the impacts of the FLF fund on their own 
skills and abilities, with 87% agreeing that they had learnt new skills on the course. 


Qualitative interviews with learners highlighted that learners entered the courses 
with different aspirations. Some learners enrolled onto courses to build soft skills 
such as improving confidence. Generally, these learners achieved their intended 
outcomes. Confidence was reported to be higher among learners who had access 
to support throughout the course, and among those who were given a tangible 
takeaway from their course at the end. 
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Another group of learners sought to gain a specific skill or qualification. The extent 
to which these learners achieved their intended outcomes was determined by the 
level of support received from employers and colleagues, as well as support 
outside of work; and the extent to which the course aligned with the reality of their 
working environment. 


The final group were pursuing longer term outcomes such as a way to work up to 
a degree, move sectors, or start their own business. This group were more likely 
to have chosen the course specifically because of its flexible format. It was very 
important for them to be able to balance their studies with their other time 
commitments for them to complete the course, and therefore continue the path to 
their desired long-term destination. 


Monitoring mechanisms: 


Providers took a variety of approaches to monitor the progress and success of 
their courses including monitoring progress online, conducting quantitative surveys 
and gathering qualitative feedback. The process of monitoring learners progress 
was perceived to be time consuming. Providers raised the need to ensure they 
were allowing adequate time for tutors to manage the process and were 
distributing the workload to reduce the burden while ensuring learners were 
receiving the encouragement needed. 


Providers learnt that they should gather feedback as the course progresses, as 
opposed to solely at the end of the course. This would increase the opportunity 
and motivation for learners to complete the feedback and give providers the 
opportunity to act on feedback and rectify any issues identified. 
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1. Introduction 


1.1 Background 


Starting this Parliament, the UK Government is providing £2.5 billion (£3 billion when 
including Barnett funding for devolved administrations) for a new National Skills Fund to 
help adults learn valuable skills and prepare for the economy of the future. It aims to 
boost productivity and help ensure that businesses can find and hire the skilled workers 
they need and help people fulfil their potential. 


In a speech on the Lifetime Skills Guarantee in September 2020, the Prime Minister 
announced that for adults who do not currently have a level 3 qualification, the National 
Skills Fund will (from April 2021) fully fund their first full Level 3, focusing on the valuable 
courses that will help them get ahead in the labour market. He also announced six digital 
bootcamps to support local regions and employers to fill in-demand vacancies by 
providing valuable skills based on employer demand. 


This announcement recognised that despite the evidence on the benefits of learning, the 
UK has recently seen a decline in the number of adults participating in learning and skills 
training.* In addition, studies have consistently demonstrated persistent patterns of 
inequality in participation.* Younger adults, people who already have higher level 
qualifications and those in higher socioeconomic classifications are most likely to be 
learning. 


The National Skills Fund also builds on extensive user research and engagement with 
local areas and employers undertaken through the National Retraining Scheme (NRS), 
which was announced in 2017 to help adults to retrain as the economy changes. The 
integration of the NRS into the wider National Skills Fund> demonstrates the 
government’s commitment to scale up a more far-reaching, ambitious adult skills offer. 


In order to test the design and provision of various training approaches for working adults 
with low or intermediate skills the Department for Education (DfE) has provided 


3 Green, F et al. (2015) “The declining volumes of workers’ training in Britain”, British Journal of Industrial 
relations 52(2) pp.422-488 


4 Learning and Work Institute (2018) Adult Participation in Learning Survey 2017 
https://assets. publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/735438/ 
Participation in Learning Survey 2017.pdf 


5 National retraining scheme: key findings paper (publishing.service.gov.uk 
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approximately £11.3 million in funding, to pilot projects that offer flexible and accessible 
ways of delivering learning to adults with low or intermediate skills who are either in work 
or seeking to return to the workplace. 


The Flexible Learning Fund (FLF) provided funding to develop courses with flexible 
delivery modes for existing courses. Funding for course delivery was provided through 
the Education and Skills Funding Agency (ESFA) budget. The FLF focused on supporting 
adults either to attain basic skills, or to progress to intermediate and higher technical 
skills, delivering teaching which took a flexible approach and by reducing situational 
barriers to learning.® 


Eligible projects either designed or enhanced a method of delivering learning that was 
not currently widely available and that was accessible to in-work adults or labour market 
returners. Since the conclusion of the Career Learning Pilot online learning provision of 
online learning has expanded across all areas of education due to the impact of the 
Covid-19 pandemic. 


Technical courses were delivered at Levels 2 to 5. Basic maths, English and digital 
courses between Entry Level 1 up to Level 2 were included. Courses delivered at degree 
level and above were not in scope. Only courses funded through ESFA were eligible.’ 


Target learners for the course were adults who were either in paid work, or were looking 
to return to the labour market following an absence, and held Level 2, 3 or 4 technical 
skills (or equivalent) and/or had yet to secure basic skills in English, Maths or digital, up 
to and including Level 2. 


Proposals were submitted by colleges and third sector organisations. Many were 
submitted by consortia or partnerships between a lead organisation and other colleges or 
employers. 


Projects were also required to meet at least one of the following “flexible delivery” criteria: 
e Delivery on a more flexible or convenient timetable. 
e Making online or blended learning work for adults. 


e Delivery outside the classroom (for example in a workplace, community centre or 
local library). 


6 


https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/656055/ 
Flexible Learning Fund - Specification for proposals.pdf (accessed March 2020) 


7 The ESFA is an executive agency sponsored by the DfE which is accountable for funding education and 
skills for children, young people and adults. 
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e Delivery methods that allow for caring responsibilities and for returning to the 
labour market (for example online teaching or teaching delivered at flexible times 
and locations). 


Of the 173 bids received, 31 projects were awarded funding based on their ability to meet 
the required eligibility criteria. One of the projects subsequently withdrew from the pilot, 
leaving 30 projects. Following a development period in 2017/18 all projects were required 
to trial delivery with learners during the 2018/19 academic year. 


The majority of successful pilot projects were provider led, with a smaller number led by 
local authorities or third sector organisations. The projects have a strong focus on 
delivering online and blended learning. Grant funding (up to £1m per project) was 
intended to be used for development/design costs and, where applicable, delivery 
enhancement costs. 


Kantar’s Public Division was appointed as the external evaluator for the FLF in 2018. 


1.2 Evaluation aims 


Four key evaluation aims were set at the outset of the FLF: 


1. To measure learner uptake and attainment compared with standard modes of 
delivery. 


2. To identify what proportion of adults enrolled on the course are the target group. 


3. To evaluate the implementation of the FLF to provide formative feedback on 
processes. 


4. To identify the scalability of different aspects of the FLF pilots. 


Multiple research questions underpin these aims, and the evaluation combines three 
strands of research designed to explore take up of the FLF, identify elements of the FLF 
that work well, and identify the areas that could be improved to ensure successful 
delivery of flexible learning provision in the future. In order to evaluate the FLF, Kantar 
collated the views of learners as well as the education providers and delivery partners. 
These views were analysed alongside learner data from the ILR and management 
information data received directly from providers. 


1.3 Methodology 


The evaluation took a mixed-method approach, using quantitative and qualitative 
research, alongside analysis of administrative data. (Figure 1) 
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Figure 1 Evaluation activities 


Qualitative strand Quantitative strand 


Questionnaire design Stakeholder workshop 


Cognitive testing 


Survey fieldwork round 1 


Wave 1 case studies 


Wave 2 case JEN 
tudi study 
Seed interviews 


Survey fieldwork round 2 


Survey fieldwork round 3 


Follow up in Produce survey dataset 
depth 
interviews 


with learners Analyse survey data 


First draft of report 


Stakeholder workshop 
Analyse ILR data Draft report 


Report finalised 


The quantitative research involved analysis of administrative data collated in the 
Individualised Learner Record (ILR)® and additional information about the courses and 
learners shared by the learning providers. The learner numbers presented are based on 
the ILR extracts (R14)° which was provided by the ESFA at the end of the 2018/19 
academic year. Providers were asked to use a numeric flag to identify learners on FLF 
courses. There were 5,362 learners flagged studying on FLF courses in the ILR extract. 
Each course offered by a provider was identified on the ILR with a Learner Aim 
Reference number. If a learner participated on more than one course, they appeared on 
ILR multiple times, once for each Learning Aim they participated on. 


8 The ILR records learner data that publicly funded colleges, training organisations, local authorities and 
employers (FE providers) must collect and share with the UK Government. 


° Please note that the R14 extract does not include all learners who were flagged as studying on an FLF 
course over the 18/19 academic year. Some learners appeared on early extracts but were removed from 
later extracts. Learners may have been removed from the extract because they were mistakenly flagged as 
being on an eligible course or because they dropped out before completing their course. 


= 
(oc) 


FLF course data from the ILR was compared with data from similar courses run in 
previous years in order to understand whether the fund increased adult learner uptake. 
Further details can be found in chapter 3. 


Kantar also conducted a mixed-mode quantitative survey with FLF learners. In total, 737 
learners out of the 5,362 identified (13.7%) were surveyed across several months 
reflecting the fact that the FLF courses ran at different times throughout the 2018/2019 
academic year. All sub-group analysis focuses on differences that are statistically 
significant at a 95 per cent confidence level. 


Telephone fieldwork was conducted between 13 March and 23 March, 19 May and 13 
June, and 27 August and 9 September 2019. Most learners were surveyed by telephone 
(584), but the option to complete online was also available, and 153 learners did so. 
Online survey invitations were issued on the 7 February, 12 March, 24 May, 15 July and 
20 September 2019. The survey link was closed on the 3 October 2019. 


Eligible learners were identified from the ILR extracts (RO3-R13) provided by the ESFA. 
All FLF learners who appeared on any extract were invited to participate in the survey. 
Survey data has been weighted to reflect the population of FLF learners recorded in the 
ILR. 


Findings from the Individual Learner Record is referred to as ILR data throughout. 
Findings taken from the learner survey are referred to as survey data. 


Alongside the quantitative research, a multi-pronged qualitative approach was taken. The 
approach enabled an in-depth understanding of the pilot at different points in its journey. 


Firstly, in-depth case studies were undertaken with eleven of the pilot providers. These 
case studies were each made up of ten telephone interviews with lead providers, delivery 
partners, tutors and learners. The exact composition of interviews was defined by the 
makeup of each case study and can be found in the sample table in Appendix A. 


The case studies were conducted over two waves. The first wave ran from January to 
March 2019 and included four of the eleven providers. This wave was conducted while 
the courses were running. The second wave of seven case studies took place after the 
courses had concluded from October to December 2019. This phase was intended to 
provide additional detail on the different methods of implementation and the subsequent 
outcomes. 


Secondly, depth telephone interviews were carried out with a representative from 15 
other lead providers not included in the case studies. The interviews aimed to gain insight 
into the implementation processes used by these providers to understand any areas of 
best practice and lessons learnt. 
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Finally, telephone interviews were conducted with learners as close to six months after 
their course had finished as possible. To allow a spread of learners to be included these 
were conducted in three groups: in October 2019, December 2019 and February 2020. 
The purpose of these interviews was to gain insight into the medium-term outcomes 
related to the course from the perspective of learners as well as exploring perceptions of 
the quality and relevance of the course. The interviews used a journey mapping 
approach to understand learner circumstances and expectations prior to the course, 
perceptions of the course, and how the course influenced learner circumstances after 
completion. 


A systematic and robust approach was taken to analysing the qualitative data collected 
across each element of the research process. The analysis was continuous (during and 
after the fieldwork periods, and between phases) and iterative, moving between the data, 
research objectives, and emerging themes. The analysis process consisted of two key 
elements. Firstly, a process-driven element using matrix mapping framework technique. 
Recordings of discussions were coded and systematically summarised into an analytical 
framework organised by issue and theme. Secondly, an interpretative element focused 
on identifying patterns within the data and undertaking sub-group analysis. Researcher 
analysis sessions, during which the team came together to discuss and test emerging 
themes and insights, were conducted after each phase and were used to support 
interpretation of the data. 


1.4 Navigating the report 


The report is structured chronologically based on the journey that providers, partners and 
learners took through the learning experience. The report pulls together findings from 
across each of the strands outlined in section 1.3. 


Throughout the report there is signposting to highlight which strand of work the findings 
refer to. The qualitative findings draw on discussions with four groups, including: 


Providers: The qualitative work draws on findings from lead providers. It 
y 4 looks to understand what worked well and less well in the set up and 
delivery of courses, and any lessons learnt that they could take forward. 


v7 | Tutors: The qualitative work also explored the views and experiences of 
ir those delivering the courses on the ground and remotely. This provides 
aaa insight into the effectiveness and sustainability of delivering a course in 


the given format. 
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bared Delivery partners: Case studies involved partners that worked with 
WY providers to design, operationalise and deliver the course where 
applicable. 


Learners: The qualitative findings incorporate the perceptions of what 
i worked well and less well for the learners during their time on flexible 
courses. 


The symbols used above are included throughout the report to highlight which group 
qualitative findings refer to. Throughout the report case studies are used to illustrate key 
points and bring the findings to life. 
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2. Programme design and implications for the 
evaluation 


Applications to the fund were open to learning providers that wanted to design, develop 
and pilot flexible learning courses for adult learners. The funding was awarded to 
providers that were able to demonstrate how they would reach the target audience 
(outlined in Section 1.1) and would adapt existing educational provision or develop new 
courses to make it more flexible and accessible for learners. 


Over the course of implementation, providers encountered some barriers that resulted in 
changes being made to their original proposals. These changes carried implications for 
the design of the evaluation which are outlined in the rest of this section. 


In their applications, providers highlighted the delivery partners (for example, other 
colleges or employers) that they would work with to deliver effective flexible learning 
courses. The qualitative interviews uncovered that, when implementing their proposals, 
providers found that they were able to make fewer meaningful partnerships than they had 
anticipated. This was largely attributed to timings, with providers suggesting that partners 
needed a longer lead in time to dedicate the necessary resource. Because of this, 
providers tended to work with fewer partners than outlined initially. 


This had a knock-on impact on the design of the evaluation. Fewer delivery partners 
were involved which meant there were fewer contacts to recruit for the case studies. 
Where possible, delivery partners were interviewed, however tutors were added to the 
sample make up at this stage. 


Providers estimated the number of learners they anticipated in their proposals. Fewer 
learners enrolled on the flexible learning courses than expected. Providers faced 
common barriers when aiming to get adults into education, for example where adults 
have had poor experiences in the past or perceive education to be time consuming. 
Recruitment difficulties in this instance were also attributed to funding delays, and 
perceived short timelines for recruiting learners. 


Kantar had planned to conduct a quantitative mixed mode survey of learners, six months 
after their course had finished. The intended purpose of the survey was to collect data to 
enable an assessment of medium-term outcomes through asking what the learner is 
currently doing (further learning, employment — new or promotion, looking for work) and 
how they feel the course impacted on these outcomes. 


Given fewer learners than expected enrolled onto flexible learning fund courses, which in 
turn resulted in a smaller baseline survey than planned, it was agreed to replace the 
follow-up learner survey with qualitative interviews. The purpose of these interviews was 
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similar to that of the survey interviews, though the follow up activity focused on a smaller 
number of more detailed conversations with learners. 
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3. Learner profile and course uptake 


The FLF aimed to develop and test flexible learning delivery approaches that would 
maximise adult participation and attainment. The target learners were: 


e Adults in paid work, or who were looking to return to the labour market following 
an absence 
and 


e Adults who held level 2,3 or 4 technical skills (or equivalent) and/or had yet to 
secure basic skills in English, maths or digital up to and including level 2. 


This chapter explores the characteristics of the learners who participated in FLF courses, 
including whether they were members of the target audience, and the extent to which the 
FLF increased learning uptake among adult learners. 


KEY FINDINGS 
1. Most learners enrolled on FLF courses were female. Two-fifths were aged 18-34 
and two-fifths were aged 35-49. Only one in five were over 50 years old. 


. According to the ILR data more than half of learners were employed and therefore 
members of the target audience. 


. Overall, 36% of learners had achieved the equivalent of a level 2 qualification or 
below prior to the start of their course, also putting them in the target audience. 


. Slightly more than half of learners surveyed reported some form of caring 
responsibility, showing that most learners were likely to be fitting study around 
other responsibilities, and potentially benefitting from flexible learning options. 


. Many projects found it difficult to accurately estimate the number of learners they 
were likely to recruit. 


. While recruitment was challenging across the project as a whole, some projects 
were more successful than others. This demonstrates that these kinds of 
programmes can be effective at increasing uptake. 


3.1 Learner profile 


Central to the FLF is the idea that flexible provision can enable a wider set of people to 
take part in further education. As such, it is expected that flexible provision should lead to 
higher uptake. 
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This section uses ILR and learner survey data to explore the demographic characteristics 
of everyone flagged in the R14 ILR extract as participating on an FLF course. 1° 


The central aim of the FLF was to make learning accessible to those who face barriers to 
traditional classroom learning. The target group was defined as low or medium skilled 
adults.'' This included those who were currently in work and those seeking to return to 
the workplace, specifically those with caring responsibilities. Among employed adults, 
those currently in insecure employment were also identified as a specific priority. 


As the ILR does not capture all the characteristics needed to define the FLF target 
groups, we have used survey data to provide additional insight into learner 
characteristics. The survey data provides an indication of the extent to which FLF 
learners were part of the target group by revealing the type of employment that survey 
respondents were engaged with prior to their course as well as their perceptions of their 
job security, their caring responsibilities and previous educational attainment level. These 
characteristics are all presented in this section. 


3.1.1 Age and gender 


As Table 1 shows, the majority of learners enrolled on FLF courses were female (65%). 


Table 1 Gender of FLF learners (ILR data) 


3,499 
Female 

65% 

1,863 
Male 

35% 


Table 2 shows the age profile of learners on the FLF courses. The majority of learners 
were under 50 (80%), while 20%% were over the age of 50. 


1° Low or medium skilled is defined as: adults that hold level 2,3 or 4 technical skills (or equivalent) and or 
have yet to secure basic skills in English, maths or digital, up to and including Level 2. 


1 Low or medium skilled is defined as: adults that hold level 2,3 or 4 technical skills (or equivalent) and or 
have yet to secure basic skills in English, maths or digital, up to and including Level 2. 
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Table 2 Age of FLF learners (ILR data) 


2,147 
18-34 

40% 

2,141 
35-49 

40% 

1,074 
50+ 

20% 


The gender and age profile of the learners who participated in the survey broadly 
matched that of the overall population of learners on the ILR. 


3.1.2 Employment status and security 


The FLF pilot courses were designed to target learners in work, and those looking to 
return to the workplace. Data from the ILR shows that the majority of learners enrolled on 
to FLF courses met this requirement. Just over half (55%) were in some form of paid 
employment or were self-employed, with 50% of this group employed, and a further 5% 
self-employed, for more than 20 hours a week. A further 30% were looking for work and 
available to start. Only one in ten (11%) did not meet this requirement, as they were 
neither in paid employment nor looking for work / available for work. (Table 3) 
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Table 3 Employment status of FLF Learners (ILR data) 


Not in paid employment, not looking for 


work and/or not available for work 


Not in paid employment, looking for 
work and available to start work 


Employed for up 20 hours a week 


Employed for more than 20 hours a 
week 


Self-employed for up to 20 hours a 
week 


Self-employed for more than 20 hours 
a week 


Unknown/missing 


576 


11% 


1622 


30% 


529 


10% 


2157 


40% 


44 


1% 


193 


4% 


241 


5% 


Similarly, the majority of survey respondents (61%) were in some form of work, part or 
full time, employed or self-employed. Figure 2 below shows the main activity of the 

learners surveyed, during the three months before starting their course. Learners were 
asked to specify their main activity based on the one they spent the most hours doing. 


(Figure 2) 
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Figure 2 Learner employment status (Survey data) 


What was your main activity in the 3 months before starting your course? 


Employed - full time (30 hours or more a Weck) [REN 37% 
Unemployed and looking for work IT 14% 
Caring for children DT 13% 
Employed - part time (between 16 and 30 hours a week) TTT 42° 
Self employed - full time (30 hours or more aweek) [NN 5% 
Employed - part time (less than 16 hours a week) [NN 4% 
Coping with a long term condition or disability BE 3% 
Self-employed - part time (between 16 and 30 hours aweek) [ll 2% 
Caring for a family member or relatively who has a disability [2% 
Self-employed - part time (less than 16 hours a week) [1% 
Getting better from a temporary illness [1% 
Looking for a training course | 0% 
In any other type of training or education BE 3% 
Other ME 4% 


Base: All learners surveyed (737) 


Learners who were in work‘? when they started their course, or during the three months 
prior to starting their course, were asked a series of questions which aimed to gauge their 
view of how secure their job was. '% 


The majority of learners who were working felt that their job had a reasonable level of 
security. Just under three quarters (74%) of those working said that they felt secure or 
very secure in their job, 16% said they felt neither secure nor insecure and 10% said they 
felt either insecure or very insecure. This is shown in Figure 3 below. 


12 In work includes both those who were employed and self-employed. 


13 Learners in work who had changed jobs since starting their course were asked these questions about the 
job they had directly before they started their course to ensure a consistent baseline measure, given that 
not all learners were surveyed at exactly the same point during their course. 
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Figure 3 Security in job (survey data) 


How secure do you feel in your job / the main job you had before starting the course? 


u Very secure = Secure = Neither secure nor insecure @ Insecure mVery insecure 


How secure do / did 6% 4% 
you feel in your job : : 


Base: All learners who were either in work when they started their or course, or who worked during the 
three months before starting their course (531) 


Most learners (62%) in work felt their job offered them good prospects for career 
progression, with 25% saying they strongly agreed this was the case. Most also agreed 
that the work they do or did in their job was meaningful (82%). (Figure 4) 


Figure 4 Perceptions of job security (survey data) 


To what extent do you agree or disagree that... 


m= Strongly agree mAgree m= Neither agree no disagree = Disagree Strongly disagree 


The job | have / had 


offers good prospects 18% 12% 8% 
for career progression 
The work | do / did 30% 41% 5% 2% 


was meaningful 


Base: All learners who were either in work when they started their or course, or who worked during the 
three months before starting their course (531) 


All learners surveyed who had ever had a job" were asked if their contract had a 
guaranteed number of hours. Three quarters (74%) said their contract or agreement with 
their employer guaranteed them a minimum number of hours, while 21% said they did 
not have a guaranteed number of hours (Figure 5). 


Figure 5 Guaranteed hours (survey data) 


Does your contract or agreement have a guarantee of a minimum number of hours? 


mYes =No = Don't know 


74% 21% 5% 


Base: All learners surveyed who had ever had a job (613) 


3.1.3 Caring responsibilities 


Learners surveyed were asked if they had any children aged under 16 or aged between 
16 and 18 and still in full-time education living with them in their household. If they 
answered yes to either question, they were classed as having childcare responsibilities. 


They were also asked if they had had any caring responsibilities for a member of their 
immediate family or a close relative. In total, 55% of those surveyed had some form of 
caring responsibilities. 


Most learners with caring responsibilities were solely responsible for looking after children 
(76%). Another 13% were both caring for their children and another relative. The 
remaining 11% of learners did not live with children in their household but had caring 
responsibilities for another member of their family or close relative. (Figure 6) 


14 Learners who were employed at the time of interview were asked about their current job. Those who 
were not working at the time of interview but had worked in the past were asked about their most recent 
job. 
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Figure 6 Distribution of caring responsibilities (survey data) 


Caring responsibilities 


= Caring for a family member or close relative but no childcare resposibilities 
= Both childcare and caring responsibilities 


= Childcare responsiblities 


11% 13% 76% 


Base: All learners with any caring responsibilities (406) 


Learners with caring responsibilities other than childcare were asked how many hours a 
week they spend looking after or helping the person they care for. For just over two-fifths 
(43%), caring responsibilities took up more than 35 hours per week, at least as much time 
as a full-time job. This included 16% who spend more than 100 hours a week caring for 
someone. 


Another 27% spent between 10 and 34 hours per week on their caring responsibilities. For 
30% caring took up less than 10 hours per week. 


3.1.4 Skill level 


The FLF projects targeted low-medium skilled adults‘® as a priority. In the survey learners 
were asked for the highest level of qualification they had achieved before starting their 
FLF course, although this did not specify the subject area. Overall, 36% of learners had 
achieved the equivalent of a level two qualification or below prior to their course, while 
17% of learners had achieved a level 3 qualification. One in ten learners reported having 
no qualifications. (Figure 7) 


15 Low or medium skilled is defined as: adults that hold level 2,3 or 4 technical skills (or equivalent) and or 
have yet to secure basic skills in English, maths or digital, up to and including Level 2 

https://assets. publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/656055/ 
Flexible Learning Fund - Specification for proposals.pdf (Accessed March 2020) 
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Figure 7 Previous qualifications (survey data) 
What is the highest-level qualification you had completed before your course? 


Entry Level qualifications (IIT o”. 
GCSEs D-G, Vocational Level 1 qualifications and equivalent (INN 10% 
5 GCSEs A*-C, Vocational Level 2 qualifications and equivalent (IN 17% 
A-levels, Vocational Level 3 qualifications and equivalent (III 17% 
Certificates of Higher Education or equivalent (IN 7 
Higher National Certificates (HNC), Diplomas (HND) or equivalent (IIT 9°: 
First degree or equivalent (II 10% 
Postgraduate qualifications — 7 
Don'tknow 9 5% 
No qualifications (I 107 


Base: All learners surveyed (737) 


3.2 Course uptake 


The FLF set out to test if providing more flexible options for adult learners could increase 
take-up of learning. This section uses ILR data to examine uptake of FLF courses in 
relation to (i) initial targets set by each project, and (ii) historic data on the number of 
learners on each FLF course in comparison to the number of learners on other similar 
courses, which aims to assess the impact of offering flexible learning options on take-up. 


3.2.1 Course uptake against project targets 


When applying to be part of the FLF, each project submitted a target for the number of 
learners who would take an FLF course.'® Table 4 outlines the number of learning aims 
funded as part of the FLF for each project against the target for each project. 


Figure 8 represents the number of FLF learning aims recorded in the ILR for each project 
as a proportion of the initial target for that project: a figure above 100% indicates that 
uptake was above the target, while a figure below 100% indicates that uptake was below 


16 More specifically, this is a target for learning aims. Each learner will have a different learning aim for 
each course that they participate on. That is, an individual learner may take more than one course which is 
part of the FLF. In conducting this analysis we have counted each learner aim individually. 
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the target. Overall, 5,362 adult learners participated in FLF courses against a combined 
target of 26,475, which was 20% of the initial total target. '” 


Recruitment of learners to the FLF was more challenging than expected for most 
projects. Four projects clearly exceeded their revised targets, and a further four projects 
were roughly in line with the target (within ten percentage points). However, for 21 
projects, uptake was below the target set. 


In some cases, uptake was far below the target; there were 13 projects where uptake 
was under half of the initial target. Chapter 6 explores some of the reasons that 
recruitment of flexible learners was more difficult than initially expected. 


Figure 8 Number of FLF learners as a proportion of project targets 
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‘7 The combined total was calculated based on the estimated number of learners each learning provider 
told Kantar they expected to recruit. 
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Table 4 Learners enrolled by project (ILR) 


Project targets Uptake: Uptake: Learning aims 
Individuals 
Lead provider Original Revised Total Uptake on Uptake on Total 
target from _ target*® number of regulated unregulated number of 
bid individual courses’? courses”? learning 
learners aims 
Access Skills Ltd 125 87 80 80 N/A 80 
Bristol Community 750 500 523 581 65 646 
Learning Bristol City 
Council 
Carlisle College 120 120 20 24 N/A 24 
Derby Business College 120 120 87 88 N/A 88 
T/A DBC Training 


18 Most providers recruited fewer learners than they expected in the first six months of the pilot. In March 2019 they shared the revised estimate of the number of 
learners they expected to recruit with Kantar, ESFA and DfE. 


18 Regulated courses are funded via the ESFA and appear on the ILR. 


20 Unregulated courses are not funded via the ESFA and do not appear on the ILR. These learner numbers were collated from communication directly with the 
learning providers. 


Lead provider 


Dudley College of 
Technology 


Durham County Council 
EDLounge Ltd. 
Fareport Training Limited 


Hounslow Adult and 
Community Education 


Hull College Group 
Inspire 


Kensington & Chelsea 
College 


Project targets 


Original Revised 
target from __ target"® 
bid 


580 580 
890 890 
3,000 800 
150 150 
700 552 
30 30 

300 200 
300 150 


Uptake: 
Individuals 


Total 


number of 
individual 


learners 


286 


105 


479 
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Uptake: Learning aims 


Uptake on 
regulated 
courses’? 


473 


229 


Uptake on 
unregulated 
courses”° 


67 


80 


Total 
number of 
learning 
aims 


474 


298 


641 


Lead provider 


London Skills & 
Development Network 


Middlesbrough College 
NA College Trust 
Newham College 
Open Awards 
PeoplePlus 


Rutland Adult Learning 
and Skills Service 


Skills and Learning 


Bournemouth, Dorset, and 


Poole 


Project targets 


Original 
target from 
bid 


270 


200 
750 
1,000 
100 
225 


200 


300 


Revised 
target’® 


270 


A 
600 
700 
100 
225 


100 


100 


Uptake: 


Individuals 


Total 
number of 
individual 
learners 


326 


43 


158 
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Uptake: Learning aims 


Uptake on 
regulated 
courses’? 


340 


43 


196 


Uptake on 
unregulated 
courses”° 


72 


N/A 
N/A 
N/A 
N/A 
N/A 


N/A 


74 


Total 
number of 
learning 
aims 


412 


43 


270 


Lead provider 


South Devon College 
St. Helen's Chamber 


The College of Haringey, 
Enfield and North East Lo 


The Consultancy Home 


Counties Ltd 


The London College of 
Beauty Therapy 


Weston College 


Workers' Educational 


Association 


Total 


Project targets 


Original 
target from 
bid 


505 
325 


180 


344 


245 


276 


700 


13,245 


Revised 
target’® 


505 
325 


120 


344 


150 


276 


700 


9,295 


Uptake: 
Individuals 


Total 
number of 
individual 
learners 
211 

159 


71 


138 


120 


300 


265 


5,357 
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Uptake: Learning aims 


Uptake on 
regulated 
courses’? 


197 
170 


95 


168 


120 


330 


101 


6,689 


Uptake on 
unregulated 
courses”? 


406 
N/A 


N/A 


N/A 


N/A 


N/A 


176 


994 


Total 
number of 
learning 
aims 


603 
170 


95 


168 


120 


330 


277 


7,683 


*National Numeracy and Open University ran ‘non-regulated’ online courses, learners on these courses are not listed on the ILR. Data on these learners has been 
collected by the projects separately. 
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3.2.2 Trends in course uptake compared to non-FLF courses 


Analysis of the ILR data showed that recruiting adult learners remains challenging, even 
for flexibly offered courses. Recruitment success varied across the projects. 


The previous section set out the number of learners on FLF projects in comparison to the 
initial project targets. However, to judge if the FLF has succeeded in its aim of boosting 
uptake, we need to estimate what would have happened without the FLF and compare 
this scenario against the actual course take-up figures. This is challenging, as the trend 
in learner numbers will be affected by many factors apart from the introduction of the 
FLF. These could be sector-wide changes (a general trend towards more/fewer learners 
taking particular kinds of courses) as well as changes specific to local areas or individual 
institutions. 


To attempt to account for these wider influences, the analysis in this section compares 
the trend in uptake for the courses supported by the FLF?' with the trend in uptake for 
other similar courses which were not supported by the FLF. There were two steps to this 
analysis: 


1. Understanding whether 2018/19 uptake on FLF-supported courses was higher than 
expected, based on uptake for similar courses which were not supported by the 
FLF. 


2. Understanding to what extent such an increase in uptake could be plausibly 
explained by the FLF. 


In practice it was not possible to include all FLF supported courses in the analysis. 
Courses were excluded from this analysis where three consecutive years of data, since 
the 2016/17 academic year, was not available. This was because with less than three 
years’ worth of data it was not possible to identify a reasonable set of comparison 
courses. Additionally, non-regulated courses (which are largely not recorded in the ILR) 
were excluded from this analysis. Finally, a small number of projects were not included in 
the analysis for the following reasons: 


e Landex recorded no FLF learners. 


e Learners on FLF courses from Carlisle College and the Capital City College Group 
made up a very small proportion of the total number of learners studying at these 
institutions (under 1%). In these cases, the analysis could not identify comparison 
courses with enough sensitivity to identify the influence of the FLF amid the 
general change in uptake year to year. 


21 |e. Courses where FLF funding was used to design flexible delivery options. 


e For four projects (Hull College Group, EdLounge, Middlesborough College, 
PeoplePlus) only one or two years of data were available. 


Step 1: Was uptake on courses supported by the FLF higher than would be 
otherwise expected? 


To estimate the change in uptake for courses supported by the FLF, this analysis draws 
on the historic trend of uptake for other similar courses which were not supported by the 
FLF. Here, ‘similar’ is defined in terms of the subject area and level of a course. For each 
course supported by the FLF, a set of non-FLF courses of the same subject area and 
level was selected as a comparison.** The comparison courses were selected so that the 
number of learners in previous years matched as closely as possible the number of 
learners on the FLF courses in previous years (before the flexible delivery options were 
introduced).*° The logic is simply that if the trend matches in previous years, this set of 
comparison courses can provide a reasonable estimate of what the uptake would have 
been for the FLF-supported courses in the absence of the FLF. 


Figure 9 shows the results for the London Skills and Development Network (LSDN) as an 
example of this analysis. 


Figure 9 Trend in uptake of FLF courses: London Skills and Development Network 
1000 


800 


600 


Baca Pa c.160 additional learners 


vs. comparison courses 


200 


198 FLF learners in 2019, 
2015 2016 2017 2018 2019 48% of all 2019 learners 


———FLF courses ===Comparison courses 


22 For example, a Level 2 Maths FLF course would be matched with non-FLF Level 2 Maths courses. 
23 More precisely, the comparison estimate is a weighted average of courses of the same subject area and 


level, following the ‘synthetic control’ method (Abadie, A., A. Diamond, and J. Hainmueller, 2010). See the 
technical appendix for further details. 
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In 2018/19, there was a sharp increase in uptake on the courses for which LSDN offered 
flexible delivery options as part of the FLF; there were almost twice as many learners 
enrolled on these courses at LSDN in 2018/19 as in the previous year (416 learners in 
2018/19, up from 232 in 2017/18, an increase of 184). 


This increase was much greater than that seen for the comparison courses (the dotted 
line in Figure 9, above); based on the comparison courses, an increase of only around 25 
learners would be expected between 2017/18 and 2018/19. In other words, there were 
approximately 160 more learners enrolled in the FLF-supported courses run by LSDN in 
2018/19 than would be expected without the FLF based on the comparison courses.”* 


Step 2: To what extent can an increase in uptake be plausibly explained by the 
FLF? 


While the method above is useful for providing an informed approximation for what 
uptake is likely to have been without the FLF, it is likely that there are important factors 
that it cannot account for. In particular, it may not be possible to control for large 
institutional level differences (for example, a large expansion of provision that is not 
related to the FLF) may lead to an increase in learners which could be misattributed to 
the FLF. 


It is therefore possible that an apparent increase in uptake could be explained by factors 
other than the FLF. It is important to recognise the overall uptake considered above is a 
combination of learners supported through the FLF and learners taking non-flexible 
versions of the same courses.” If the number of learners recruited through the FLF is 
large, then it is plausible that the FLF has contributed to an increase in uptake. On the 
other hand, if the number of learners recruited through the FLF is small, any increase in 
uptake will be largely due to other factors. 


In general, the increase in uptake that can be attributed to the FLF should be less than 
the total number of FLF learners recruited. This is because some of these learners would 
still have taken a course without the FLF. Therefore, the number of FLF learners 
recruited represents an upper bound for the impact of the FLF on course uptake. 


24 This is the difference between the number of learners enrolled on the FLF-supported courses in 2018/19 
at LSDN (416) and the expected number of learners for 2018/19 based on the comparison courses (257). 


25 For many courses supported by the FLF, learners had the option to take a non-flexible form of the same 


course. These were generally courses which had been offered for a number of years by a provider, and so 
already had established models of delivery outside of the FLF. 
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In the example of LSDN (see Figure 9, above): 198 FLF learners were recruited in 
2018/19. While with the data available it is impossible to be certain how many of the FLF 
learners would have taken a course without the FLF, this is large enough to plausibly 
explain the increase in uptake: 


e Overall, there were about 160 more learners on these courses than would be 
expected based on the comparison courses. 


e The number of FLF learners (198) was larger than this. It is therefore plausible 
that the FLF led to an increase in uptake, even after allowing for the possibility that 
some of these learners would have taken a course without the FLF. 


There were seven other projects with a similar pattern (see Appendix 1). Specifically, 
these are projects where (i) uptake was higher than expected (based on uptake of the 
comparison courses), and (ii) the number of FLF learners was large enough that it is very 
likely the FLF contributed to the increase in uptake (even allowing for the possibility that 
some of the FLF learners would have still taken a course without the FLF). 


Across these eight projects (see Appendix 1), the FLF could plausibly have led to around 
1,000 more learners taking a course than would have been the case without the FLF. 


Aside from the eight projects referenced above, there were ten other FLF projects where 
uptake in these courses in 2018/19 was higher than expected based on the comparison 
courses (Figure 10). However, in these cases it was less clear that the increase in uptake 
was associated with the FLF. This was because the number of FLF learners recruited 
was considerably smaller than the increase in uptake. As an example, for Dudley College 
of Technology, there were around 580 more learners than the comparison courses in 
2018/19, but there were only 266 learners recruited for the FLF on these courses. In 
other words, while it is plausible that the fund made some contribution to the increase in 
uptake, there must also be other important factors. 
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Figure 10 Trend in uptake of FLF courses: Projects where the number of FLF 
learners was smaller than the estimated increase in uptake. 


25000 Dudley College of Technology 
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Finally, there were five projects where there was little or no evidence for a substantial 
increase in uptake. (Figure 11) In these cases, the actual number of learners in 2018/19 
was very similar to the estimated number of learners from the comparison courses. There 
is therefore little reason to think the fund had a major impact on uptake for the FLF- 
supported courses in these projects. 
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Figure 11 Trend in uptake of FLF courses: Projects with little or no increase in 
uptake 
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While the overall data shows that recruiting adult learners remains challenging, even for 
flexibly offered courses, there is some evidence from comparisons with non-FLF courses 
the that the FLF led to a substantial increase in uptake for courses across eight projects. 
This demonstrates that that these kinds of programmes can be effective at increasing 
uptake. The factors that contributed to successful recruitment are discussed in chapter 6. 


It is also likely that the FLF contributed to increased uptake in courses across up to ten 
other projects. However, in these cases there must also be other important factors 
influencing uptake. 


However, there were five projects where there is little or no evidence that the FLF led to 
increased uptake for these courses. 


It is important to recognise that some courses within projects were not covered in this 
analysis (primarily where there was only one or two years of data available for courses of 
that subject area and level within a project). It is possible that some projects were 
successful in widening access for these additional courses which cannot be included in 
the analysis presented here. 


The factors which contributed to successful learner recruitment are discussed in Chapter 
6. 
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3.3 National Numeracy and Open University courses 


National Numeracy and the Open University both received funding through the FLF. As 
most of their learners participated on non-regulated courses offered freely online, they 
were not included in the quantitative analysis. Both organisations conducted evaluations 
of their own projects. The findings are briefly outlined below.° 


3.3.1 National Numeracy 


National Numeracy is an independent charity which aims to raise low levels of numeracy. 
In 2014 they launched online provision to improve basic numeracy skills that are needed 
in the workplace. The provision encourages adults to engage with an online assessment 
or “skills challenge” which identifies an appropriate learning target. The learner is then 
signposted to third-party online educational resources.?’ 


For the FLF project, National Numeracy set out to look at the effectiveness of using this 
form of online learning to engage working adults in improving their basic maths skills. The 
FLF allowed National Numeracy to widen their approach, particularly in the area of using 
digital platforms to reach working adults directly, or by working with specific employers 
and to assess the effectiveness of different approaches. 


National Numeracy originally aimed to attract 15,000 working adults to engage with their 
online provision. In total 59,423 adults engaged with the project exceeding the target. 
Around one-third of these (20,040) were in work and had no plans to take a maths 
qualification at the time of engagement. 


National Numeracy tested a variety of online approaches to attract learners to participate. 
They found that it cost less than £6 per head to engage adults online via Facebook, 
Google search and Bing. They found that Facebook was the most effective way to 
engage adults at scale. 


They also worked with employers to engage learners. Recruitment was most successful 
when staff across all levels were committed to the programme, particularly department 


26 Please note that in addition to recruiting learners via their unregulated provision national Numeracy 
recruited 170 learners onto regulated courses. Open University recruited 62 learners onto regulated 
courses. 


27 National Numeracy, (‘Using online learning to engage working adults in improving their basic maths 
skills’ A National Retraining Scheme ‘Flexible Learning Fund’ research project, Unpublished. 
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managers. However, securing high levels of commitment was challenging. Most National 
Numeracy participants accessed the tool independently rather than via their employer. 


3.3.2 Open University 


The Open University “Bringing Learning to Life” project was funded through the FLF to 
offer functional skills courses in English and maths through the OpenLearn online 
learning platform which hosts the Open University’s openly licensed learning materials. 
The Open University worked with The Bedford College Group, Middlesbrough College 
and West Herts College to develop online courses and validate the learning through 
qualifications. 


The project offered Levels 1 and 2 English and maths with the course content made 
available online. There were two classes of learner within the project. Informal online 
learners took one of the four OpenLearn courses without taking an exam. They were 
given the opportunity to gain a ‘soft’ certificate to indicate they had undertaken the 
course. Formal learners took the course online and registered for a related formal exam 
at one of the partner colleges. 


According to their communication with Kantar, the Open University expected around 
11,000 learners to complete the course informally. According to the Open University 
evaluation, in the first seven months after the launch of the programme, 1,871 learners 
achieved an “Everyday Skills” badge after completing the course online. They also 
expected 540 learners to register formally with one of the three partnering colleges; 62 
learners participated formally. 


Further details are available in the Open University’s evaluation of the “Bringing Learning 
to Life” project. 2° The evaluation also includes learner survey data, including some 
demographic questions from the national evaluation survey. 


8 Farrow, Robert (2019). Bringing Learning to Life: Evaluation of Everyday Skills in maths and English. In: 


Open Education 2019, 30 Oct 2019-1 Nov 2019, Phoenix, AZ. 
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4. Designing and adapting the course 


This section uses data from the qualitative case studies and the learner survey to explore 
the process that providers went through to design, adapt and operationalise flexible 
learning courses. First, it highlights that courses were better designed where course 
objectives were effectively and efficiently adapted. Second, it explores the key 
approaches to adapting course content including streamlining, tailoring and collating 
content. Finally, it outlines how communication and time management were key to 
ensuring course content was digitised effectively. 


Golden Rules for designing and adapting the course 


Course objectives and content should be adapted to fit the flexible delivery mode, 
for example through reducing content to ensure the material can be covered in the 
time or signposting to key content. Learner needs should be front and centre. 


Where possible, course objectives and content should be linked to specific sectors 
or occupations to ensure learners can see the relevance to the workplace 
environment. 


Engage stakeholders such as employers, colleges or experts in developing 
objectives and content as early as possible in the process to maximise buy in and 
impact. 


Allow time to develop clear briefs for delivery partners, and for an iterative 
approach to course content to ensure improvements can be made throughout the 
process. 


Providers that had established relationships with digital developers were had a 
smoother experience of designing and adapting their courses due to a knowledge 
of how developers worked. The Department for Education may need to focus on 
capability building among providers, particularly in relation to digital capability to 
ensure providers are able to best use the resource available. 


4.1 Adapting course objectives 


When reflecting on how to adapt a course to be flexible, providers initially 
focused on the course objectives. Generally, they recognised that objectives 
should evolve to reflect the changing nature of course delivery and the needs of 
the target audience. 
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The intended outcome of a course was a factor in determining the extent to which 
providers adapted objectives. Where a Flexible Learning Fund course led to a recognised 
qualification, a level of consistency with the existing traditionally delivered course was 
needed. To pass the qualification, learners needed to build the same knowledge 
irrespective of how the course was being delivered. 


In these cases, where courses led to a recognised qualification, some providers 
acknowledged the need to make minor adjustments to objectives. This was more 
common among providers that invested more time and resource into upfront course 
design, where instead of directly transferring objectives from more traditional 
mechanisms of delivery, they tailored them to the needs of the audience. For example, 
some providers set out how objectives should be prioritised to highlight key areas to time- 
poor learners. 


Learners on courses where providers had made adjustments were generally 

ws more positive about the volume of work. Conversely, in cases where providers 
had not accounted for learner needs when adapting objectives, both learners and 
providers referenced the density of the course. 


“The content was chunky [too heavy in volume] ... because it was a 
basic course, | didn't think it would be as chunky as it was.” Learner, 
Wave 2 


Providers who ran courses for recognised qualifications also reported better 
anil engagement and outcomes when the objectives were linked to a target sector. 
While objectives remained largely consistent with the course they were being adapted 
from, nuance was added to ensure relevance for leaners. This was only possible where 
the provider was targeting learners from a specific sector. For example, the objectives of 
one IT skills course were adapted to relate specifically to how they could apply in the 
health and social care sector. 


Other providers ran courses that were not tied to recognised qualifications. Instead they 
were designed to meet the needs of specific employers or sectors. For example, 
research conducted by one provider identified a skill shortage in the local area. The 
provider designed the course objectives with the aims of filling the local skills gap. 


Providers leading these courses had more flexibility to adapt course objectives. There 
was greater confidence in the revised course objectives where they were developed in 
collaboration with delivery partners. Generally, providers that involved their partners at 
this stage held workshops or roundtables to ensure all parties could engage with the 
process fully. Others had more light-touch collaboration where partners were given the 
opportunity to comment on objectives developed by the lead provider. 
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“We feel that the best products are created by getting the 
stakeholders around the table” Lead, Wave 2 


Although most providers were positive about this process, a small number discussed 
some negative impacts of collaboration. Some providers perceived including multiple 
stakeholders in objective development to be time consuming and reported that the time 
spent on objective development left them with little time for other essentials. For one 
provider, the time spent on developing objectives was felt to have reduced time 
available for recruitment and their learner numbers were significantly lower than 
expected as a result. 


4.2 Adapting course content 


Providers recognised that course content needed to be adapted to ensure it was 
suitable for the target audience. They took different approaches to establish 
relevant and high-quality content for learners. These approaches included: 


e Streamlining content; 
e Tailoring the content to a specific sector; 


e Collating resources within a consortium. 


Streamlining content 


The most common approach was to streamline content. It took higher priority among 
those that were moving more content online, those with more substantial resource and 
those who described themselves as having a more detailed knowledge of the existing 
content. These providers recognised a need to reduce the base amount of content that 
learners would have to process, to ensure they remained engaged. 


Providers that streamlined their content reviewed the courses they wanted to make 
flexible. They identified the core topics while ensuring the content remained aligned with 
the specific qualification or partner needs. Providers reported that going through this 
process made them more focused on priorities, therefore they were able to target 
material development more efficiently. This is demonstrated below in Case Study 1. 
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“We sat down, me the Vice Principal and the Programme Managers 
and went through what was essential to cover and what could be 
slimmed down. We used that information to know what we needed to 
convert into online content or videos.” Lead, Wave 2 


Providers that did not streamline content generally perceived themselves to be time-poor. 
They considered the process of launching the course to be rushed and so did not think 
they had time to review and cut content. A small number of these providers outsourced 
content development to an organisation with less knowledge of the specific course aims 
or audience. Because of this, they were less able to prioritise. Where course content was 
not streamlined, both learners and providers suggested that the content was too dense. 


Learners surveyed who said the course did not meet their expectations mentioned that 
the course was rushed. This shows that learner experience may be improved by ensuring 
that courses cover an appropriate volume of material. 


Tailoring course content 


<_ “Ss An alternative approach to adapting courses was to tailor the content. This 
eS was found among providers that were aiming to build capacity. Some 
providers were looking to build capacity for specific partner employers or organisations. 
In these cases, the two parties worked together to identify skills gaps that a course could 
look to fill. In other cases, providers had used research about the local area to identify a 
misalignment between the skills of the local population and local employers. 


“We wanted to respond to the wider community...one way to do that 
was to bring colleges and providers together to work collectively with 
the Combined Authority to identify where the gaps are and to provide 
a more holistic solution for the area.” Lead, Wave 1 


Providers tailoring course content in this way highlighted the positive influence on their 
relationship with employers. Where there were existing relationships, these were 
maintained and consolidated. Providers reported positive feedback they had received 
from employers. In circumstances where providers had targeted skills gaps more broadly, 
they referenced new relationships they had built with local employers through their 
courses. 


“It was really helpful to be consulted, engaged. It wasn't this is what 
we're doing, what do you think? It was what do you need?” Delivery 
partner, Wave 2 
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A small number of providers mentioned that they would have liked to tailor their course 
towards a specific sector but did not, identifying a lack of time available at an early stage 
as a key barrier. 


Collating course content 


Less commonly, providers that were working in a consortium adapted the course 
“ content by collating their resources with partner organisations. The aim was to 
identify the highest quality resources and combine them to produce a superior course. 
This was perceived to be a time-consuming process, and providers suggested that a 
more effective course could have been produced should they have had more time. 
However, providers thought it was a beneficial process and would continue to work in this 
way. 


“We wanted to align our materials to be contextually embedded into 
local priorities, but there was limited time to make this work. It’s what 
we're working on doing going forward.” Delivery partner, Wave 1 


Achieving streamlined, tailored or collated content 


There were two overarching ways in which providers streamlined, tailored or collated 
content: partnership working and iterative development processes. 


gs Providers commonly collaborated with expert delivery partners to adapt 

‘ the content of existing courses. These experts were employers in the sector 
their courses were aimed at, local colleges that had identified a need for more flexibility 
for their learners, or other providers. 


Adapting course content through partnership working had mixed levels of success. The 
partner's understanding of both the provider organisation and the learners they were 
adapting content for played a determining role in the success of the partnership. Both 
providers and partners highlighted the need for a clear briefing at the outset. As seen in 
Case Study 1, where this had happened, fewer disruptions in the process were 
highlighted. 


Where briefings were unclear or felt rushed, partners raised confusion over what their 
role involved. This was echoed by providers who pinpointed occasions in which partners 
had not delivered what was expected. For example, one provider commissioned a 
partner to streamline their content for an online platform. The partner misunderstood their 
role and simply transferred all the content online without prioritising and reducing it. This 
impacted the density and therefore the user experience of the course. 
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Partners with subject-specific expertise were perceived to have a greater depth of 
understanding of learner needs. In these instances, providers had more confidence in 
their work which meant less time was dedicated to overseeing what they were doing. For 
example, one provider worked with a videographer who had expertise in both 
videography and the course subject. They were able to capture the important elements 
for the learner with minimal oversight from the provider, while making the end product 
engaging and visually stimulating. 


“We ran a mini tendering process and commissioned someone who 
had expertise in the subject as well as being able to create a great 
engaging product. We had the best of both worlds.” Lead, Wave 2 


Where there were existing relationships between the provider and the partner, more 
comprehensive working relationships were common, with high levels of trust from 
partners that the course would be created with their needs in mind. Processes were also 
in place to minimise the burden on both sides. 


In a small number of cases, the partnership was less successful despite having an 
existing relationship. Time pressures were cited for the reduced level of partner 
engagement in adapting content. Partners stressed that they were unable to meet the 
resourcing needs at short notice, and often had to get through levels of bureaucracy to 
get approval for involvement. 


Where providers were building new relationships there was more work to be done to 
create buy-in. Providers not only had to convey the benefit for the partner, but also build 
processes and a level of trust in a short period of time. Providers who did this 
successfully held briefing meetings, or workshops at the outset of the relationship to 
ensure all parties could feed into the ways of working and were bought in to the direction 
of the course. Providers again suggested that short timescales were the key inhibitor to 
holding these briefings and workshops. 


Another approach taken to streamline, tailor and collate content was to adopt an 

iterative development process. Providers with more resource dedicated to course 
development intended to adapt their course content iteratively from the outset. The aim 
was to continually improve the course content over time, through feedback from tutors 
and learners. This resulted in more refined content for learners that generally received 
positive feedback. 


However, providers found this approach to be resource intensive. In some cases, 
dedicated staff were placed under significant pressure to deliver, with working hours 
much longer than anticipated. In other cases, additional staff were required to work on 
course development. 
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“It just took a lot longer than expected. The whole thing was stressful 
for everyone involved and it meant it was all hands-on deck.” Non- 
case study lead 


Where less internal resource had initially been assigned, course development was 
sometimes reactive. These providers had anticipated having all content complete before 
the course was launched to learners, but then found that course development was a 
longer process than anticipated and had to continue development as the course was 
running. This generally caused high levels of stress for providers and some suggested 
that this led to lower levels of course quality than originally hoped. 


Case study 1. A positive multifaceted approach to adapting course content 


London College of Beauty Blended learning for beauty therapy. Used online resources 
for theory, video demonstrations and exams and classroom 


Therapy (LCBT) learning for practical skills. 


LCBT partnered with an e-learning provider with the aim of adapting course content to be digestible in an online 
format, and a videographer to provide online demonstrations before practical classroom learning 


At the outset, the core team ran through existing courses they 
wanted to make flexible. They highlighted which elements of the 
course were essential and prioritised these in the online content. 
Other areas were given as additional reading. 


Siigr- lanl ifaliare mece)alcciale 


LCBT partnered with an e-learning provider. There were mis- 
communications about how innovative they could be, based on 
their budget as well as the extent of the partners involvement in 
converting materials to be suitable for an online platform. 


Defining roles at the 
fo} 0) Y=) we) j 
partnerships 


Partners that Following a tendering process, LCBT selected a videographer to 
create online demonstrations of different practices. They chose 
someone with expertise in beauty as well as videography. The 
outcomes were informative and well produced. 


understand learner 
needs 


4.3 Digitising the course 


Although not all providers used digital content, most had taken steps to digitise in 
— some way. This section focuses on providers that had moved some or all their 
course content online. There were two key factors that impacted the extent to which 
digitising a course was perceived to be a success: communication and accurate time 
management. The effectiveness of both communication and time management were 
related to the previous experience of working with, and the relationship with, platform 
developers. 
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Communication 


Providers had varying levels of experience of creating digitalised courses. For a small 
number of providers, Open University for example, digitised content was commonly used. 
In these instances, providers had mature and ongoing partnerships with platform 
developers, and established processes creating a shared understanding of what was 
needed to successfully digitalise a course. Positive experiences were also common when 
providers worked with an in-house platform developer. Both parties noted that working 
within the same organisation gave them an understanding of each other’s needs. Course 
leads knew the information that developers would need from them, and developers had a 
greater understanding of the course needs. 


More commonly, providers were new to digitalised content or were using a new platform 
developer. Providers suggested that communication needed to be more rigorous in these 
circumstances, with the establishment of core expectations at the outset a priority. 
Providers highlighted instances of platform providers misunderstanding needs where 
clear briefings had not been held, leading to additional and unintended work for 
providers. Where expectations were clarified effectively at the outset, platform developers 
were able to flag where they were unable to meet some of the needs. 


“We thought they’d be able to take material and convert it, but they 
actually wanted us to do this for them so they could just put it online. 
We were working evenings and weekends, it was a really hard three 
months.” Lead, Wave 1 


An iterative approach to communication was also a factor in the success of digitalising 
the course. Where communication was regular and ongoing, issues were raised earlier. 
For example, one provider was not satisfied with the quality of the look and feel of the 
platform but, as they had taken an iterative approach, they had time to push back and get 
changes made. 


Accurate time management 


Providers that were using existing platforms for the pilot were able to manage their time 
well. They had gone through the process before, often multiple times, and so knew how 
to manage time effectively. 


There were mixed levels of effectiveness in time management where providers were 
using in-house developers. Where they were already engaging with developers regularly, 
both parties were more aware of their roles. 
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“Well we’ve done this process many times. We work with [the Digital 
Learning Manager] all the time so we know what's what.” Non case 
study lead 


However, providers who did not regularly use digital content tended to underestimate the 
amount of resource needed to translate a course onto an online platform. For example, 
there were cases where course leads did not build in time to test the platform both 
internally and with learners, leading to concerns about the quality of the product. 


Providers raised concerns about how effectively they could scale up flexible learning 
courses because of the time needed to test both internally and with learners. If the 
number of courses delivered flexibly were to increase, additional resource would be 
needed to manage the testing process. While providers were able to see the benefits of 
investing resource into this work in the long-term, it was commonly not resource that was 
available to them. 


Case study 2. Effective digitisation of course content 


Online or blended learning based on business enterprise, 
with the aim of helping learners thrive in self-employment. 


PeoplePlus 


PeoplePlus created traditional paper workbooks, tested them with learners and then translated them into online 
learning. There were different stages of testing and adapting content. 


The operational team recognised the importance of working 
Oey a alan elalcer=1iceyal closely with the technical team. The day-to-day lead of the 
project was brought in to bridge the gap between the teams. 


ed =o) e)(-\ ed [ULM — 10) Mal Kelu(e|amaalelit|e)(-m-jt-(e(-\-me) me{-yV/-1(e]0)aq(-vaim (e) 
Iterative achieve the final online course. The technical team created a 
development beta version, which was tested and then adapted to the alpha 

version. 


Accurate time management allowed time for tutors to test the 
online content with some initial learners. This did take a lot of 
time and there were some concerns about doing this on a larger 
scale for more courses. 


Time management 
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5. Learners’ motivations and experience on the course 


This section uses the learner survey data to explore learners’ self-reported experiences 
of learning on an FLF course, including motivations for choosing their course, the 
benefits of flexible learning options and barriers which prevented learners form 
completing their course. 


Kantar interviewed 737 learners out of the 5,362 identified (13.7%) on the ILR. Survey 
data has been weighted to reflect the population of FLF learners recorded in the ILR. All 
sub-group analysis focuses on differences that are statistically significant at a 95 per cent 
confidence level. Please note that in some cases results may not sum to 100% due to 
rounding. 


Key findings 


e Career progression was a key motivating factor for many learners. Younger 
learners were more ambitious for career advancement while older learners were 
more likely to say they wanted to improve in the job they had. 


Most learners said that at least some of their teaching was delivered outside of 
a school or FE setting. 


While online only options were taken up by fewer than half of survey 
respondents, the online approach appears to have been important in allowing 
them to participate. 


A sizable minority chose a flexible course to fit around their existing 
responsibilities. This was particularly important for women indicating that 
flexibility was valued by learners and enabled them to fit studying around their 
other responsibilities. 


5.1 Motivations for learning 


This section explores learners’ general motivations for studying as well as their reasons 
for choosing a flexible course. 


Career progression was a key motivating factor for many learners. Almost three-quarters 
(73%) of learners surveyed said that they enrolled on the course for work or career 
related reasons. Those studying on Entry Level courses were least likely to say they 
were studying for work related reasons (54%) while those studying on Level 3 or other 
courses were most likely to be studying to further their career (89%). 
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Those who were working during the three months prior to starting their course (and said 
that they were studying for work related reasons) were given a list of options and asked 
to choose the one that best described why they had enrolled. More than half (54%) 
wanted to develop or improve in that job. Another 7% were studying as they wanted to 
get a promotion in their current line of work. Other learners were motivated to study by 
the prospect of a career change. Just over two in ten (23%) said they signed up for their 
course as they wanted to retrain into a substantially different type of job. 


Among those who were working in the three months before they started studying, the 
youngest learners appeared to be the most ambitious for career advancement. Learners 
in the 16-34 age group were the most likely (13%) to be studying to get a promotion in 
their current line of work. Only 4% of 35-49s and 3% of those aged over 50 were 
specifically studying as they wanted a promotion. 


However, the youngest group were less likely than those aged 35-49 or 50+ to say they 
wanted to develop or improve in the job they had (45% compared to 59% and 61% 
respectively). 


Learners aged 50+ were the least likely group to say that they were studying because 
they wanted to retrain (14%) in comparison to 28% of 35-49-year olds and 25% of 16- 
34s. This could reflect the fact that as workers in later stages of their careers, they were 
already well-established in their jobs, or felt that they were too old to retrain. 


Most FLF learners surveyed were able to study free of charge. Only 15% said that there 
was a fee for the course which was not covered by government funding. Of those whose 
course was not fully funded by the government, 38% said the fee was covered by their 
employer. Just over a third (35%) said they paid the fee directly from their own money. 
Some learners took out an advanced learner loan (10%) or borrowed money from family 
or friends (5%). (Figure 12) 
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Figure 12 Sources of non-government funding for FLF fees (survey data) 


Who paid the fee? 
Employer covered some or all the costs a) 38% 
Paid fee directly from own money airs] 35% 
Took out an advanced learner loan | 10% 
Borrowed money from friends or family | 5% 
Took out another form of loan | 1% 
Some other way || 6% 
Don't know aa 5% 


Prefer not to say l] 1% 


Base: All learners surveyed who took a course not fully funded by the government (116) 


5.2 Reasons for choosing a flexible course 


Learners surveyed were also asked for their main reason for choosing a flexible course. 
The results indicate that flexibility was valued by learners and enabled them to fit 
studying around their other responsibilities. Overall, 39% of learners said that they had 
chosen their course in order to fit studying around their responsibilities.2? Women (43%) 
were more likely than men (33%) to say that they chose a flexible course to fit around 
their existing responsibilities. 


Twenty-two percent of learners said they had chosen a flexible course to fit around their 
work schedule. This was particularly the case among those whose course was 
completely online (44%) compared with those whose course was only partly online (21%) 
or had no online elements (13%). 


In addition, 10% said they had chosen it to fit around their childcare responsibilities, 2% 
to fit studying around their caring responsibilities and a further 5% to fit studying around 
other responsibilities they had. Learners aged over 50 (4%) were less likely than learners 
aged 35-49 (14%) or 16-34 (11%) to say they had chosen a flexible course to fit around 
childcare responsibilities. 


A full breakdown of the main reasons learners gave for choosing their course can be 
found below in Figure 13. 


° This included all learners who said they chose their course to fit around work, caring responsibilities for 
children or other family members, or any other responsibilities they had. 
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Figure 13 Main reason learners chose a flexible course (survey data) 


What was the main reason you chose your course? 


To fit studying around work schedule 

The fact that it was online made no difference 
To learn at own pace 

To fit studying around child care responsibilties 
It was not online 

There wasn't an alternative offline course available 
To fit studying around other responsibilities 

To better self/increase knowledge 

To fit around your caring responsibilities 

It was mandatory/did not chose the course 

To learn/improve English language 

For (future) employment 

Other 

Not stated 


Don't know 


Base: All learners surveyed (737) 


5.3 Flexible delivery 


nn 22°, 
es 14% 
es 13% 
ee 10% 
ee 9% 
a 6% 

a 5% 

Ma 3% 

mm 2% 

Ma 2% 

Mm 2% 

mm 2% 

ee 8% 

ma 2% 

mi% 


Survey respondents were asked a series of questions to establish how the course was 
delivered. Around half (52%) said that their course was delivered ‘in person’ only, 27% 
said their course was delivered through a mix of online and ‘in person’ teaching and 22% 
said their course was delivered online only. (Figure 14) 


Figure 14 Course delivery mode (survey data) 


How is the majority of your course delivered? 


mln person’ only 


52% 


Base: All learners surveyed (737) 


= Through a mix of online and ‘in person’ teaching 


= Online only 


27% 22% 


Learners whose course involved any ‘in person’ teaching were asked where that learning 
took place. As these courses were flexible in nature, learners were able to select more 


than one answer. 


In keeping with the FLF’s aims to trial flexible options to fit around learners’ other 
responsibilities, 62% of ‘in person’ learners said that at least some of the teaching was 
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delivered outside of a school or FE setting. Forty percent of learners took part in learning 
at a school or college. (Figure 15) 


Figure 15 Location of ‘in person’ teaching (survey data) 


Where does the face-to-face learning take place? 


At a training centre or the office of the course provider (So aes 30% 


At a community centre 13% 


At your place of work ana 13% 
At a library ‘| 6% 


Other, namely... a) 2% 


Base: All learners whose courses involved a face-to-face element (570) 


Note: Learners were able to select more than one response. 


All learners surveyed on courses with an online element were asked whether they would 
have been able to take part in the same course if it were delivered entirely in person, in 
order to understand whether online delivery allowed participants who would not otherwise 
have signed up for a course to take part. 


Although less than half of learners took part in a course with online teaching, these 
learners found the online delivery very helpful. More than half (57%) of learners whose 
course was completely online said that they would not have been able to do the same 
course if it were delivered entirely in person during the day. Another quarter (26%) said 
they would only have been able to participate in-person, during the day, with difficulty 
(Figure 16). 


Only a third (34%) of learners on courses with any online element said they could 
definitely have done their course in-person during the day. A similar number said they 
could definitely have done their course in-person during the evening (32%). 


In comparison, those who were participating in a course which included both an online 
and offline element were more likely to say that they could participate in a course that 
was completely in-person. However, this would still prove to be a challenge for a sizable 
proportion of these learners. Just under a quarter (22%) would have been able to 
participate during the day with difficulty, whilst 38% said that they could definitely not 
participate. 
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Evening courses would also have been difficult for learners who took part in courses 
taught exclusively online. Thirty-eight percent said they would not have been able to take 
part in an in-person evening course and a third (34%) said that they would only have 
been able to participate in person with difficulty. Of those whose course was partly online, 
43% definitely could not participate in an evening course, and 23% could only take part 
with difficulty. 


Attending a daytime course would be equally difficult for those both with and without 
caring responsibilities, but those with caring responsibilities (27%) were less likely than 
those without caring responsibilities (38%) to say they could attend an evening course. 


Conversely, there was little difference between the proportion of employed and 
unemployed learners who said they would be able to participate in an evening course. 
However, a larger proportion of those who were not employed (53%) said they would 
definitely have been able to take part in a course during the day than those who were 
employed (25%). 


Figure 16 Ability to participate in course if delivered in person only (survey data) 


At the time you enrolled on your course, would you have been able to do the same course if... 


Yes, definitely m=Yes, with difficulty =No 


It were an evening course delivered entirely in It were a day time course delivered entirely in 
person? person? 


Base: All learners surveyed whose course was partly or completely online (367) 


Learners who said they would not have been able to participate in person either during 
the day or evening were asked why they would not have been able to do the course. 
Their responses were unprompted and coded once fieldwork had been completed. Half 
(51%) said that work commitments would prevent them from participating in an entirely 
face-to-face course. Childcare (30%) or other family commitments (10%) were also 
common barriers. The full list of barriers is shown in Figure 17. 


62 


Figure 17 Reasons learners would not be able to participate in less flexible course 
(survey data) 


Why wouldn't you have been able to do the course? 


Work commitments i 51% 
Child care commitments (ITN 30% 
Family (care) commitments (all other references) SN 10% 
Transport/travel/location distance issues =| 8% 
Health reasons [i 4% 
| have other commitments/do other things [ij 3% 
Would prefer daytime course fi 3% 
Spouse/partner work commitments B 2% 
Time commitment [fj 2% 
Study commitments [J 2% 
Inconvenient/more difficult [J 2% 
Financial issues [J 1% 


Would not want to (no detail) [J 1% 


Base: All learners surveyed who would find it difficult to commit to a daytime or evening course delivered 
entirely in person (297) 


5.4 Other flexible options 


Learners surveyed were asked about the study options, resources and facilities that were 
available to them and which they used during the course. Their responses show that they 
appreciated being able to get in touch with their tutors easily. Most learners were able to 
get in touch with their tutor via email (87%) or in person (83%) with a large majority (72%) 
choosing to communicate with the tutors using these methods as shown in Figure 18. 


A smaller, but still sizable, proportion said they were able to contact their tutor by 
telephone (67%). 


Only a quarter of learners had the option to communicate with tutors directly via Skype or 
an online chat function. However, online communication seems to have been less 
popular with learners than other methods. Only 12% of learners surveyed had contacted 
their tutor via an online chat function. 


Most providers offered online resources (76%) such as learning materials, workbooks 
and reading materials. These were used by 67% of all learners. Fewer learners said that 
their provider offered online teaching (30%) either via video or live using applications 
such as Skype. Only half of these learners said that they had engaged with teaching 
delivered online. 
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Figure 18 Flexible options available (survey data) 


As part of your course did you have access to any of the following? 


= Available m= Used 


A tutor you could communicate with directly via email Pe a 
P 83% 
A tutor you could see in person I 
; 76% 
Online resources as. 


A tutor you could contact by telephone 44% 87% 


A peer support network organised by the learning provider | re 51% 


Teaching delivered online 16% 30% 


A tutor you could communicate with directly online via skype | 25% 
or a chat function 12% 


Base: All learners surveyed (737) 


Learners who said that they were able to get in touch with a tutor directly by any method 
were asked when tutors were available. Although courses were designed to be flexible 
and accessible for people juggling multiple responsibilities, only 38% of learners were 
able to get in touch with their tutor outside of typical office hours. Almost all learners 
(85%) were able to contact their teachers between 9am and 6pm. (Figure 19) 


Figure 19 Availability of teaching staff (survey data) 
As part of your course are teachers available... 


= Don't know mNo mYes 


... between Yam and 6pm ...before 9am or after 6pm 


Base: All learners were able to get in touch with teachers or tutors (712) 


Learners were also asked about the resources and facilities that were offered by 
providers. More than half (58%) of providers gave learners access to equipment, such as 
laptops, to help them complete their studies. Learners seem to have found the option to 
borrow equipment helpful, with 46% of all learners surveyed saying that they used these 
resources (Figure 20). 


Providers also commonly made spaces for independent study available to learners. 
Thirty-nine percent of providers allowed access to a study space in the evening, early 
mornings or at the weekend. These spaces were used by 29% of all learners surveyed. 
Younger learners aged 16-34 (38%) were more likely to use study spaces than those 
aged 50+ (21%). 


Only 16% of providers offered childcare facilities or support. Despite the high proportion 
of learners with childcare responsibilities, only 5% of learners used these facilities. Even 
among learners living with children, only 8% used childcare facilities. Another 11% said 
that they had not used childcare facilities, even though they were available. Use of 
childcare facilities was more common among learners with children aged 0-3 (25%). 


Figure 20 Facilities provided (survey data) 


And did your course provider offer any of the following facilities, even if you didn’t use them? 


m Available Used 


; 58% 
Access to equipment such as laptops 
46% 


Access to a room or study space to study independently - | eae | 39% 
available in the evenings or early in the morning or at the 


weekend 29% 


Creche or other child care support 


5% 


Base: All learners surveyed (737) 


The 55 survey respondents (8%) who did not complete their course were asked why not, 
they were able to specify more than one reason.°° Respondents were given a list of pre- 
coded reasons they could choose from but were able to specify other reasons. The most 


30 Please note, these results are based on a small number of responses and, as such, should be taken as 
indicative only. 
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common reason given for leaving the course was that it was too difficult to juggle 
studying with work (26%), childcare (12%) or caring commitments (9%). Some learners 
dropped out because they changed their mind about their career aspirations (9%) so the 
course was no longer relevant to them. 


Some non-completers cited issues with the course itself as reasons for dropping out. This 
included 7% who said they did not like the course, 7% who said that it was poor quality 
and 3% said that there were problems with the technology used to deliver the course. 
The full list of reasons given is shown in Figure 21. 


Figure 21 Main reasons learners did not complete the course (survey data) 


What were the main reasons you didn't complete the course? 


Couldn't juggle studying with my job DEE TTT 26% 
Couldn't juggle studying with childcare DE 12% 
The amount of time / hours was too much EE 12% 
Changed mind about future career/type of job TT, 9% 
Couldn't juggle with my caring commitments EE 9% 
Poor health DT! 8% 
Didn't like the course — course content not of interest E 7% 
Poor quality EE 7% 
Other commitments EE 7% 
Found the course too hard EN 7% 
Financial reasons ME 5% 
Didn't receive enough support from the learning provider [A 4% 
The course wasnt what! expected itto be ME 3% 
Problems with the technology / the technology did not work ME 3% 
Too much technology / found the technology too complicated ME 3% 


Base: All learners surveyed who did not complete the course (55) 


Note: Due to small base size results are indicative only. 
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6. Recruitment and retention 


This section begins by exploring how learners found out about flexible courses, and the 
level of success experienced by providers. It then explores four methods of recruitment, 
using qualitative data to identify where these were most successful and where there were 
lessons learnt. These four methods included: 


e Recruiting through partnerships; 
e Enquiries and word of mouth; 

e Digital marketing; and 

e Offline marketing. 


It also explores provider strategies for retaining learner engagement once they were on 
the course. 


Golden Rules for Recruitment and Retention 


Providers should build an understanding of their audience and target their recruitment 
communications to more effectively attract learners. Messaging should be 
personalised where possible. 

DfE should publicise the benefits of flexible learning to both potential providers and 
partners. Understanding the benefits of flexible learning should increase buy in and 
ensure courses are being advocated for. 

Providers should engage with partners, including employers, early in the process to 
facilitate recruitment. They should be mindful of timings to avoid partners having 
competing priorities and to allow space to build relationships. 

According to the learner survey, social media did not have a high success rate for 
recruiting learners. In qualitative discussions, providers attributed this to lack of 
targeting of the adverts. Where adverts were more targeted, more traction was found. 
Tutors and other individuals with direct contact with potential learners require ongoing 
support. This should include a full brief on the flexible learning options and how to give 
recommendations to engage learners, as well as ongoing support with using the 
technology to facilitate continued engagement. 

Dedicated and ongoing tutor support was key to learner retention. Flexible learning 
formats made this more time-consuming for tutors. Should courses scale up, providers 
would need to ensure a greater number of tutors were bought into the benefits of 
flexible working to ensure they dedicate the required time to learners. 
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6.1 Approaches to recruitment 


Providers generally used a multi-pronged recruitment approach in efforts to reach their 
target number of learners, using existing marketing approaches, partnerships and new 
advertising campaigns to attract learners. 


The learners surveyed were asked how they found out about the courses. The largest 
group found out about the courses through a current or past employer (30%), suggesting 
that partnerships with employers have helped FLF providers recruit learners who would 
not otherwise have signed up for a course. (Figure 22) 


Learners who were studying courses delivered entirely online were most likely to say that 
they found out about the course through their employer (58%). Only 17% of those 
studying a course with no online elements and 29% of those studying on a partly online 
course found out about the course through their employer. 


Almost one in five (18%) learners heard about their course through friends or family 
members. A similar proportion (17%) found out about their course through a Government 
or Public Sector organisation (including local schools), Jobcentre Plus or the National 
Careers Service or Next Step. 


Figure 22 Where learners found out about the courses (survey data) 
Where did you first hear about the course? 


Your current/previous employer (ITN 30% 
Friends or family SN 18% 
Through the college or learning providers website (IN 11% 
Jobcentre Plus EE 10% 
Google or other web search engines SN 10% 
Government/Public sector organisation EN 4% 
National Careers Service, Careers Advisor or Next Step EE 3% 
Other advert ME 3% 
Third sector organisation BE 2% 
Through the college offline [ij 2% 
Facebook/social media [i 1% 
Other, namely... EE 4% 
Don'tknow MI 1% 


Base: All learners surveyed (737) 


Despite this multi-pronged approach, the ILR analysis showed that recruitment was 
commonly problematic. Engaging adults in learning is widely acknowledged to be a 
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challenging task, with multiple barriers such as in the low levels of uptake.*" In these 
circumstances, providers also attributed difficult recruitment to short timescales and lower 
than anticipated engagement from partners. 


The projects were classified into three groups (Table 5). 


Table 5 Recruitment outcomes *2 


Lead provider*? 


seitesaleiley:\aejm@e aan London Skills and Development Network; Bristol Community 
learners was at Learning; Derby Business College; Fareport Training Ltd.; NA 
least as great as College; South Devon College; The London College of Beauty 
the estimated Therapy; The Consultancy Home Counties Ltd. 

increase in uptake 

(oy-t-Y-To Mela Mul ol f-1.¢-1 

of the comparison 

courses) 

shiewilile-meogea aa Dudley College of Technology; Hounslow Adult and 
learners was Community Learning; National Numeracy; Open Awards; Skills 
smaller than the and Learning Bournemouth, Dorset and Poole; St Helen’s 

CES iuiriicveMlneag-s-ti-a) Chamber; Access Skills Ltd.; Durham County Council; 

in uptake (based Inspire; Weston College 


on uptake of the 
comparison 
courses) 


at (-Wo) am ale) Kensington and Chelsea College; Newham College; Rutland 
Talei¢:t--Mlameleelcwe Adult Learning and Skills Service; The Open University; 
(oy-t-y-To Weyaullejt-)<(-t Worker’s Educational Association 

of the comparison 

courses) 


31 https://learningandwork.org.uk/resources/research-and-reports/adult-participation-in-learning-survey- 
2019/ 


33 Providers listed in bold took part in the case studies. 
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The case studies showed that providers with a smaller than estimated increase in uptake 
commonly focussed on encouraging existing learners or enquirers to consider a flexible 
course. They also promoted different types of flexible courses including fast-track options 
and allowing learners to choose both when and where they study. 


The providers with little or no increase in uptake attributed the challenges to a late launch 
due to delays in developing flexible course content. 


6.2 Recruiting through partnerships 


Providers commonly worked with partners to facilitate the recruitment process. 
These partnerships included a wide variety of organisations including: employers, 
colleges, charities and local government. 


As noted above, recruitment through employers was the most common route. Providers 
targeted potential partners based on their links to the target audience, and the objectives 
of the course. For example, where courses focused on a specific sector, they 
approached employers in that sector who may want to upskill staff. 


Providers running courses for academic qualifications more commonly looked to partner 
with colleges, with the aim of being able to offer an alternative course format to their 
learners. Some organisations such as charities and government bodies were approached 
because of their ability to access harder to reach groups, such as unemployed 
individuals. 


Recruiting through partnerships had mixed levels of success. The outcomes were largely 
dependent on the partner’s knowledge of the target audience, the extent to which the 
partner understood the benefit to them, and the maturity of the relationship between the 
two parties. 


Partner knowledge of the target audience 


. A key determinant of the success of recruiting through partnerships was the extent 
“¥ to which the partner understood the target audience. It was essential that they 
recognised which learners would be capable of working in the given flexible format and 
how to engage these learners with the idea of the course. 


Where most successful, partners knew the types of learners that would be most suited to 
a flexible learning approach. They were able to proactively reach out to individuals that 
they were already working with to offer them the opportunity of signing up for a flexible 
course. Their existing knowledge of individual circumstances enabled them to target their 
communications effectively to increase the likelihood of further engagement. These 
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partners were also able to react to new enquiries, and direct those most suited to flexible 
learning towards the provider. 


Other partners were less far along in this journey but were beginning to recognise the 
importance of understanding the specific circumstances of those they were working with. 
For example, one partner college outlined recent Ofsted feedback that suggested many 
of their learners would benefit from alternative approaches to education, including flexible 
courses. These providers were looking to action this feedback through trying to better 
align learning format to individual needs. 


Where partners, or potential partners, did not understand the target audience to the same 
extent, the provider had more difficulty in engaging them in a meaningful way. 


A common rationale given by providers for not reaching their estimated number 
of learners was that partners recruited fewer than expected. In these cases, 
providers thought that partners had not targeted learner recruitment. Instead they had put 
out broad messages to large cohorts. Without personalising the recruitment drive, they 

were less able to engage the right learners effectively. 


“[The partner college] put out a message but | don’t think it really 
resonated. | don’t think the tutors really knew much about it and who 
it was for.” Non-case study lead 


Providers also referenced organisations that they had approached for partnership, but 
that had turned them down. While in some instances, this was simply because they did 
not have enough time or resource to dedicate to the partnership, others did not recognise 
that they might work with people for whom flexible learning might be relevant. 


Partner understanding of the benefit 


laa Recruiting via partnerships was perceived to be more successful where the 

mm partner was able to identify tangible benefits of recruiting learners. Where 
partners recognised the benefit to them as an organisation, they invested more resource 
and time into making the partnership fruitful and ultimately got more learners signed up to 
courses. 


Where partnering was a success for colleges, they identified two core benefits to working 
with the provider. First, it was perceived to attract a more diverse range of learners to the 
college. Having flexible courses available opened learning to a broader target audience 
including shift workers and individuals with caring responsibilities. Colleges perceived this 
to increase their own learner numbers and profile within the sector. Second, on some 
existing courses, colleges thought that the partnership had improved the quality of 


71 


teaching resources. Ultimately, they thought that this could be a method of improving 
results. 


Employers were better able to understand the benefits to their employees if they were 
involved in course development from the outset. Their involvement enabled them to 
shape the content of the course to better fit their needs and to upskill their employees in 
the most useful ways. In these cases, they acted as an advocate for the provider, 
encouraging or even making it compulsory for employees to complete the course. 


On some occasions, charities, government bodies and service providers also partnered 
with providers to recruit learners. They were most engaged when they had identified how 
courses could help customers learn skills that would ultimately reduce the burden on 
them as an organisation. For example, one partner recognised that a functional skills 
course could move customers into employment or more secure employment, ultimately 
reducing the burden placed on them. 


Where partners were less able to see direct benefits to their organisation, they were 
slower to engage and did not advocate as strongly for learner engagement. This was 
found where providers did not engage partners at the initial stages of course 
development, with courses therefore less aligned to the needs of the partner and with 
fewer direct benefits. 


Engaging with potential partners at a later stage also gave them less time to understand 
the core benefits and dedicate resources to promoting the courses. In some cases, late 
engagement left strategic leaders with less time to engage managers to promote flexible 
learning to employees. As a result, the course did not become embedded in company 
culture as a priority. In other cases, leaving engagement until after course development 
meant providers were trying to engage employers in seasonal peaks, when employers 
were more reluctant to engage fully or release staff for learning. 


Maturity of the relationship 


ee % The extent to which providers had previously worked with individual 

"partners influenced how engaged partners were in the recruitment process. 
While prior engagement was useful in facilitating a smooth relationship, it was not 
essential. 


Providers that had previously worked with local partners on other programmes or 
initiatives generally looked to build on these relationships as part of the pilot. The relevant 
individuals within each organisation had built a level of cooperation and trust over time. 
As a result, facilitating engagement between the organisations was a smoother process. 
For example, one provider was able to outline the meetings that would be needed with 
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specific individuals to get the project moving. They had a good understanding of partner 
priorities and so less time was needed to convince partners of the benefits of 
collaboration. 


“We've worked together on multiple bids before, so we all knew what 
was what.” Lead, Wave 2 


Where relationships were new, more groundwork was needed to allow for successful 
partner recruitment. Providers recognised that where time was invested in involving 
partners at the start of the process, there was more buy in. Some referred to steering 
groups or round tables that made the project feel like a collaborative effort, and not a 
favour to the provider. 


Some partners were less open to investing this time up front. They did not know what the 
outcomes would be, having never worked with the providers before, so were willing to 
devote less time to the project. This was more common where the parties were based in 
different locations as they found it harder to engage with partners that they could not 
regularly meet face-to-face. 


Engagement levels were also lower where providers were simply too late in initiating 
contact. Some attributed this to the delay in funding, meaning less time was available to 
forge meaningful relationships. However, more commonly providers did not have enough 
resource to dedicate to building relationships at the outset when other priorities, such as 
developing course content, were prioritised. 


Case study 3. Effective recruitment through partnerships 


An online portal offering courses in a range of subjects 
Weston College including: leadership and management, enterprise skills, 
IT user skills, tenancy and autism awareness. 


Weston College worked closely with a range of partners who promoted their courses to employees and 
customers. The relationship with Alliance Homes was particularly successful. 


Alliance Homes worked closely with its customers, and had a 
depth of understanding of which of their customers had skills 
gaps that the course could fill. 


Understanding of the 
target audience 


Alliance Homes identified that the course offered by Weston 
College could help upskill customers towards employment, 
creating more sustainable tenancies. 


UFitelJecieclatelinlemeym tals 
benefit 


Weston College and Alliance Homes had a strong and long- 
Long-standing standing relationship having worked together previously on 
relationship numerous bids. They understood each other's ways of working 
and trusted each other. 
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Case study 4. Effective recruitment through partnerships 


Blended learning courses delivered by a range of colleges 
Dudley College of Technology - in the area. Aims to attract learners who may have lost 
Improving Access to Skills for the Region motivation to attend traditional courses. 


A college within the consortium worked with a local employer to identify a skills gap in the workforce and develop 
training to help employees build their knowledge in this area. 


Many of the employer's services were moving online. They 
identified that many employees would not have the necessary 
skills to use them effectively. They recognised that the course 


Partner 
Platel=iscitslaleliale me) mul 


benefits would facilitate better engagement with the tools once rolled out. 


Partner knowledge The employer was able to identify which employees had lower IT 
of the target skills levels and was able to approach them and sell the benefits 
audience to them in a personalised way. They identified 600 employees. 


Delays in provider access to software has meant they were 
Timings unable to simulate the online content before it went live, meaning 
it could not be shown to leamers beforehand. 


6.3 Enquiries and word of mouth 


P74 Tal All providers interviewed mentioned directing those who were making 
222 general enquiries towards flexible learning courses where they 
considered it appropriate. Although relatively few learners surveyed 
mentioned that they first heard about the course through direct enquiry, either online or in 
person, providers perceived it to be a useful strategy with many citing it as one of the key 
engagement methods. There were a number of circumstances in which providers would 
do this: 


e Tutors recommending flexible courses to learners already enrolled on other 
courses; 

e Referring potential learners that had made enquiries about courses over the 
phone or online; and 

e Referrals at open days. 


Providers and tutors suggested that having direct contact with potential learners ensured 


they could give a full explanation of the course details, as well as assuaging any potential 
fears of the learner. 
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“When people called up to ask, we’d briefed everyone on the flexible 
courses so they could direct them if it felt right.” Lead provider, Wave 
2 


Moreover, learners that enrolled through this strategy were, to a certain extent, already 
engaged. Those who were already enrolled on another course had been exposed to 
adult education and were familiar with the processes they needed to go through to make 
the course work for them. Alternatively, learners who were making enquiries had already 
recognised that they wanted to take part in a course and were looking for the ‘best fit’ for 
them. To engage with ‘harder to reach’ learners, providers needed to employ the other 
recruitment methods outlined in this report. 


As flexible learning was new for most providers, engaging potential learners in this way 
tended to be an informal process, with referrals made at the discretion of tutors and 
administrative staff as and when they felt it appropriate. Providers recognised that there 
were likely to be inconsistencies in how this was being rolled out. 


One provider had taken steps to embed the flexible learning offer more formally in their 
course brochure to make the process consistent. In the brochure, the provider had 
outlined the level of flexibility of each course. While it was too early to draw findings 
about the impact this had on uptake, the provider was optimistic about increasing learner 
exposure to the offer. 


Despite initial positive outcomes from embedding the flexible learning offer into regular 
enquiries, there were some disadvantages to the approach. Using this strategy alone did 
not enable providers to reach out to hard-to-reach individuals but simply signposted 
those who were already bought into the idea of adult education. 


eS As well as hearing about courses from provider themselves, almost one in five 

“=” learners surveyed had been introduced to courses by friends or colleagues. 
Again, in the qualitative interviews these individuals tended to be those who were already 
thinking about signing up to additional training. 


“Someone from work was telling me about something similar she’d 
done, and | thought oh | should probably do something like that.” 
Learner, Wave 1 


6.4 Digital marketing 


Digital marketing was a recruitment method employed by most providers in the 
«& pilot. A variety of channels were used to reach learners online including providers’ 
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own websites, Facebook Ads, Google Ads and some more targeted spaces such as 
Mumsnet. As mentioned previously, the learner survey showed mixed results for digital 
recruitment with 10% of learners saying they first heard about the course through Google 
or another search engine. Advertising through social media appeared to be less 
successful; only 1% of learners surveyed reported first hearing about their course 
through social media. 


Providers generally thought of digital marketing as a tool for raising the profile of and 
building interest in the course. Some providers referenced a high ‘click-rate’ for adverts. 


However, there was generally less success in converting initial interest into enrolment 
statistics. Providers that had analysed data had found that a high proportion of learners 
that had clicked on the advert were not eligible for the course. These providers 
suggested that there was further work to be done in ensuring adverts were targeted to 
the correct audience. 


“There was a lot of interest to start with but that wasn’t seen in our 
final numbers.” Non-case study lead 


In contrast, where providers had published their adverts in more targeted spaces there 
had been more success. For example, one provider identified that many potential 
learners would use Mumsnet. The advert posted on the site gained lots of traction and 
was a key route to successful recruitment. The provider therefore suggested that 
targeted digital marketing was more successful. 


Providers recognised that digital marketing would be less successful in reaching digitally 
excluded audiences and therefore often triangulated this approach alongside others. 
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Case study 5. Targeted digital marketing 


London Skills and Development A blended approach offering courses in a range of 
subjects including railway engineering, health and social 
Network care, English, Maths, fitness instructing and childcare. 


London Skills and Development Network marketed their courses online. Mumsnet and Reeds courses were key 
platforms that were used. 


Targeted marketing 


SJUlit=]e)(=m ce)mm e)(=yale(=1e| 
learning 


Triangulated with 
other methods 


6.5 Offline marketing 


Providers tended to complement their digital marketing with offline marketing to 
all reach out to digitally excluded learners. This took various formats including 
leaflets and posters that were distributed at different locations in the local community, 
such as community centres and neighbourhood events. Rather than a core method used, 
offline marketing tended to be used in conjunction with other approaches. 


Providers were often uncertain about how successful these methods were and rarely had 
reliable methods for measuring how many learners had come in through this route. In 
less residential areas, providers felt that they lacked a natural community to advertise to 
and recruitment through these channels was perceived to be less effective. These types 
of advertising accounted for 3% of the learners surveyed signing up for their course. 


“We were encouraged to advertise in the community. But we’re in the 
[city] centre, there is no real sense of community in the same way 
you’d get elsewhere. That just wasn’t an option for us.” Lead, Wave 2 


6.6 Retention strategies 


As well as encouraging learners to enrol on courses, providers also used a range 
of strategies to continue learner engagement during the course and reduce drop- 
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out rates. A key element of this strategy was ensuring communication and engagement 
with learners at different timepoints during the course. 


More established providers, and those working with employers, generally recognised the 
need for communication with learners between sign-up and the start of the course. More 
established providers sent simple reminders to keep learners warm to the prospect of 
starting the course and minimise dropouts. Where employers had recruited on the 
provider's behalf, employers were able to reinforce the importance of the course and the 
logistics involved when learners were at work. 


Providers perceived continued learner engagement to be more successful when an 
induction was held at the outset of the course. The introduction was conducted either 
face-to-face or online and tended to include an overview of the course, outline 
expectations and, where applicable, demonstrate how to use the relevant software. 


Learner feedback about these introductory sessions was more positive when they 
ws were detailed and held face-to-face, with a series of common themes emerging 
about the benefits of this approach: 


e An increased sense of feeling valued; 
e Increased learner motivation; and 
e Agreater level of comfort in interacting with technology. 


Firstly, differences emerged in how valued the learner felt as they started the course. 
Personalised service was key to satisfaction. Where courses offered a face-to-face 
introduction, learners expressed a greater sense of being valued by the provider. There 
were more concerns raised by those that did not receive a face-to-face introduction that 
they were ‘another number’ and that the provider did not value them as individual 
learners. Learners whose courses were entirely online suggested that they would have 
also benefited from a level of personalisation at the outset rather than experiencing the 
more generic introduction that many experienced. This was consistent with the survey 
findings that satisfaction levels were higher on courses with a face-to-face element. 


Learners who received a formal face-to-face introduction also expressed greater 
motivation towards the course. The session was perceived as a symbolic starting point 
and marking it with a face to face event was perceived to frame the course as important. 


“We were told there would be a meeting at the start where they’d talk 
us through everything and of course this didn’t go ahead. It was really 
disappointing. It’s carried on in the same way... lots of promises but 
nothing actually delivered.” Learner, Wave 2 
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Where there was an online element to the course, most providers used the introductory 
session to familiarise learners with the software and increase ease of use for the 
remainder of the course. Face-to-face sessions allowed tutors to demonstrate use more 
effectively and resulted in learners reporting a greater level of comfort in interacting with 
the technology. 


“| went there, and they set me up with a username and password and 
run through all the basics...1 was good to go.” Learner, Wave 2 


Learners also suggested that provider engagement needed to continue throughout the 
course. Where courses were delivered face-to-face, this was a natural process, as tutors 
engaged with learners each time they attended the course. However, online course 
providers had to put alternative strategies in place, including: setting up in-person drop-in 
centres; allowing learners to contact tutors through the platform or Skype; monitoring 
learner progress data and contacting those struggling; and proactive feedback from 
tutors. 


As in the introductory stage, contact throughout the course increased the learner's sense 
of being valued and increased motivation. Learners varied in the level of interaction they 
wanted with providers, but regularly discussed feeling valued where communication was 
consistent with what the provider outlined at the start of their course. They recognised the 
time and effort that was being committed by the provider and felt further engaged with the 
course as a result. When providers did not engage with learners throughout the course, 
or engaged less than was promised at the outset, learners reported feeling less 
motivated to continue with the course. 


“| emailed for help and got nothing. | felt like they weren't interested 
in me to be honest.” Learner, Wave 1 


o/] As previously discussed, higher levels of access to tutors correlated with higher 
pre levels of satisfaction with the teaching among the learners surveyed. The 
qualitative interviews also showed that tutors played an instrumental role in the extent to 
which learners were engaged and were encouraged to continue their courses. They 
varied in the extent to which they were motivated and able to do this. Motivation was 
determined by the extent to which tutors had previous experience of similar ways of 
working and by the individual characteristics of tutors. 


Tutors were more consistently motivated in organisations that had some background of 
using technology in learning. Some providers had experience of using similar platforms 
for other projects and therefore tutors had been exposed to using them in the past. For 
others, technology used for flexible learning engagement had been introduced over time. 
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For example, one provider had already been using tablets in face-to-face sessions to 
stimulate learning. 


“It wasn't like this was a whole new thing for us. We'd used the 
tablets in other courses, so | knew how to use that.” Tutor, Wave 2 


Where tutors had been working for the same provider and delivering courses in the same 
way for a long time, they were less motivated to change their behaviour. In these 
organisations, strategic leaders allocated responsibility for delivering courses to specific 
tutors who were more motivated and were perceived to be high performing. 


These tutors wanted to perform well and dedicated a lot of time to learner engagement. 
They noted that learners responded well with personalised support, a view that strategic 
leaders within provider organisations also noticed in learner feedback. However, tutors 
raised concerns about the sustainability of this level of input should the courses be 
scaled up. While personalised mentoring was commonly still being advertised and 
advocated for by the provider, there was a concern that this would not be possible should 
the volume of learners increase. 


Providers emphasised that additional tutors would be needed to support flexible courses 
should they continue. They suggested a need for a top down approach to increasing tutor 
motivation and that senior leadership would need to be fully bought in to the vision for 
motivation to trickle down. 


“In the future, we’re going to need to find some others to work on the 
courses. That will take a lot more work and preparation time.” 
Delivery partner, Wave 2 


While some tutors were motivated to deliver the pilot courses, there were some concerns 
raised about whether they had the relevant knowledge to do so effectively. Where tutors 
were using technology that was new to them, they were more confident in doing so 
where they had received training and guidance at the outset and throughout. 


It was common for tutors to receive a briefing before the course began on how to use the 
technology to engage with learners. Tutors were more positive where there was 
assistance throughout the course, for example through handbooks on how to use the 
technology, or a point of contact to reach out to with questions. 


Where briefings and continued support were not offered, or were of low quality, there 
were more inconsistencies in how learners experienced the support offered. Providers 
were generally aware of this where it was the case, as they had received feedback from 
both learners and tutors highlighting difficulties. One provider was already putting steps in 
place to tackle the issue through developing a course on best practice for online learning, 
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aimed at tutors, in order to make sure all staff are fully engaged and able to effectively 
deliver and monitor their courses. 


Aside from helping to motivate and guide learners throughout the course itself, tutors 
were commonly also responsive to learners’ external circumstances. For example, in one 
case a tutor helped a learner facing housing issues. 
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7. Course delivery 


This section explores learners’ expectations of course delivery, the extent to which they 
were met and providers’ understanding of why course delivery was perceived in that way. 
It begins by exploring learners’ overall satisfaction with the course. It then looks at the 
quality of course content, suggesting that content was perceived to be higher quality 
where it was tailored, of an appropriate volume and complexity and where supporting 
materials had a professional look and feel. It then considers the role of personalised, 
responsive and consistent teaching quality in driving learner perceptions. Finally, it 
outlines the ways in which, and the extent to which, providers ensured content was 
accessible. 


Golden Rules for Course Delivery 


Content should be tailored to real-world scenarios, including situations that learners 
would find themselves facing at work or in everyday life. Working with partners with 
knowledge of working environments can facilitate this. 

The volume and complexity of content should be clearly defined at the outset. 
Providers should be mindful that learners may have been out of education for 
significant amounts of time and should plan volume and complexity accordingly. 
Although providers spent more time adapting courses for online delivery, learners on 
courses with an in-person element seemed to be more satisfied than those learning 
solely online. 

Learners have high expectations of online content. Online materials should be high 
quality and add value to the learning experience. The best results were achieved 
where developers collaborated with subject experts. 

Tutor flexibility was expected. Learners expected tutors to be available outside of 
traditional working hours to be consistent with their working times. 

The learner survey showed that the presence of a supportive tutor was the most 
common reason the course exceeded learners’ expectations. Qualitative findings 
reinforced this idea, suggesting that personalised support was paramount to a 
positive experience. 

To maintain high levels of trust between learners and providers, providers should 
ensure that they keep commitments made when advertising the courses. 
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7.1 Overall learner satisfaction 


Learners surveyed were asked what they thought of the course, overall, based on their 
experience. Just over one-third (37%) said that the course was better than they 
expected. (Figure 23) 
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Figure 23 Course expectations (survey data) 


Based on your experience so far, overall, would you say that your course is/was... 


= Better than expected = About as good as expected m= Worse than expected Don't know 


Base: All learners surveyed (737) 


Learners were given an opportunity to explain why the course exceeded their 
expectations. These were unprompted responses which were coded once fieldwork was 
complete. Common responses included the teaching staff being helpful/supportive and 
the course being informative or enjoyable (20% respectively). The full list of reasons 
given are listed in Figure 24. 


Figure 24 Reasons course exceeded expectations (survey data) 
Why does/did your course exceed your expectations? 


It was informative/I learned (a lot) from it DIT 20% 
The teaching staff were helpful/supportive IIT 20% 
Happy with the course/I enjoyed it/better than | expected IT! 20% 
The teaching was good/good quality I 17% 
It was more in depth/challenging DN 15% 
The teaching staff were friendly/approachable [NN 6% 
It was easier than anticipated [A 6% 
| have used what I've learnt (at work, athome) ME 5% 
Positive experience of other students MEN 5% 
Passed the course/got a qualification BNE 5% 
| liked studying online/it was easier to study online than | expected MN 3% 
Other DE 10% 
None © 1% 
Don'tknow # 1% 


Base: All learners surveyed whose course was better than expected (253) 


Note: Learners were able to give more than one response. 


The 10% of learners who said that the course was worse than expected mentioned that 
courses were rushed and reported technical difficulties with online learning platforms, 
poor quality learning materials and lack of support from tutors. (Figure 25) 


Figure 25 Reasons course did not meet expectations (Survey data) 
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Why does/did your course not meet your expectations? 


Lack of support from teaching staff 25% 


Problems with online teaching provision 21% 


The course was rushed 15% 


12% 


Not relevant (to me)/not what | was expecting 


Other 


26% 


None 


2% 


Base: All learners surveyed whose course was worse than expected (92) 


Note: Learners were able to give more than one response. Due to small base sizes results are indicative 
only. 


7.2 Quality of course content 


Learners’ expectations of course content varied. Those without prior experience 
Ss of adult learning, or who had been asked to enrol on the course by their employer, 
generally had not given much consideration to the content. They therefore started the 
course with few expectations of what would be included. Those who had previously taken 
part in adult learning started the course with some expectations of what would be 
covered. They thought that the course would give them the basic skills needed for their 
career development, or in an area of interest. 


The learner survey asked participants about their satisfaction with specific aspects of 
their course. 


Learners reported positive views of the quality of various aspects of the courses, but a 
higher proportion were satisfied with the quality of the teaching delivered offline (90%) 
than the quality of the teaching delivered online (78%). (Figure 26) 


A similar percentage of learners were satisfied with the quality of the offline (88%) and 
online (87%) learning materials used. 


Learners aged between 16 and 34 were more likely to say they were satisfied with the 
quality of the content of the offline learning materials (91%) than those aged 35-49 
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(85%). The youngest learners, aged 16-34, (91%) were also more likely than those aged 
35-49 (83%) to be satisfied with the quality of the online learning materials. 


As noted earlier, the FLF courses were designed to focus on flexible and distance 
learning options. In many cases they focused on online alternatives to classroom 
delivery. However, despite this focus on online options, learners on courses with an in- 
person element seemed to be more satisfied than those learning solely online. Learners 
who were studying on a course that was delivered entirely online were less satisfied with 
both the online teaching (60%) and learning materials (70%) than those who were 
learning on a mixed online and offline course (81% and 89% respectively). 


A higher proportion of ‘Entry level’ (93%) and ‘Level 1’ (95%) learners were satisfied with 
the in person teaching than those taking ‘Level 2’ (88%) or ‘Level 3 or other’ courses 
(81%). 


Figure 26 Learner satisfaction with course (survey data) 


How satisfied were you with the quality of... 


u Very satisfied u Fairly satisfied = Neither satisfied nor dissatisfied u Fairly dissatisfied u Very dissatisfied 
The teaching you received in person 64% 25% 4%3%3% 


Any offline learning materials 55% 34% 8% 3% 


Mea 5% 8H 5394 
materials 53% 34% 8% 5%3% 


The teaching delivered online 44% 34% EE 


Base: All learners surveyed (737) 


Similarly, the qualitative interviews showed that learners’ expectations of course content 
were largely met, although some areas of improvement were identified. Important factors 
determining learner satisfaction were: 


e The extent to which content was relevant and tailored; 
e The volume and complexity of the content; and 
e The extent to which supporting materials were considered professional. 


Relevant and tailored content 


Learners were more satisfied with course content when they could apply it to real- 
ws world scenarios. For those referred to a course by their employer, this was found 
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where the content reflected issues specific to their industry, imparting knowledge that 
they could use directly in their job roles. For example, learners from one provider 
discussed how they had been taught how to use email and digital calendars more 
effectively. This had enabled more efficient communication in their workplace. 


"The workbooks were good. The content was relevant to my job on 
the whole." Learner, Wave 2 


Learners who were not currently in employment also welcomed skills that could be 
applied in their lives. Where learners were looking to get into employment after course 
completion, they were positive about skills that they thought would be applicable in their 
desired sector, or that were pre-requisites for specific jobs. Other learners were not 
looking to get into employment as an outcome of their course. The skills this group 
wanted to gain were those they could use recreationally or were functional skills to 
facilitate easier everyday life. 


“Before when it said ‘IT’ I'd think ‘oh no’, but now I'd be ok with that - 
I'd apply. | feel like I've definitely moved forward with it.” Learner, 
Wave 2 


ee’ Providers echoed this sentiment. Feedback from learners reinforced the need to 

adapt content to make it more relevant. They were aware of the need to 
continually tweak the materials to include more work-related scenarios or use further 
examples that reflect students’ everyday lives. 


Providers who had strong relationships with partners tended to have more tailored and 
relevant content. Having strong relationships was reported to facilitate more open 
communication about the needs of the target audience. This enabled providers to build 
on their existing expertise and effectively shape course materials. Providers with weaker 
partnerships, or that did not have time to build relationships received more feedback that 
the course did not feel relevant. 


Volume and complexity 


The volume and complexity of course content influenced /Jearner satisfaction. Too 
much or overly difficult work was perceived to impact morale for a variety of 
reasons. 


First, learners tended to refer to other priorities in their lives, including caring 
responsibilities, paid work or spending time with their families. They wanted the volume 
and complexity of the work to feel attainable and digestible without having to significantly 
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alter their lives. Second, learners who had negative experiences of previous education 
lacked confidence in their ability to achieve if the content was seen as too heavy. They 
suggested that content would need to be digestible for them to maintain good morale and 
ultimately to complete the course. 


Providers who successfully delivered digestible content did so by bringing 
«together key members of staff to discuss what they wanted the course to cover. 
Once this was agreed, it was tested with learners and further adapted. 


Similarly, providers who managed to pitch the content at the right level of complexity 
were those who were responsive to the learners’ needs and abilities and changed 
content accordingly. Students struggling with content were more likely to need additional 
support, so for providers to avoid incurring extra costs in both time and money, getting 
the level right was vital. 


"Each provider had their own courses and curriculum. Some were 
tried and tested more than others." Lead, Wave 2 


Professional supporting materials 


Learners expected course content to look polished and professionally designed 
7 as a minimum standard. Where online content was used, they expected platforms 
to include interactive features such as quizzes and forums, and to be able to save and 
upload their assignments easily via the portal. As this was seen as a minimum standard, 
learners interviewed tended only to comment on materials where they were deemed to 
be low quality. They remarked on where there were glitches in the online content 
meaning work was not saved, or where the materials were of a basic level and lacked 
interactivity. 


While learners expected innovative and interactive features, they wanted them to serve a 
recognisable purpose rather than being used as an ‘add on’. Learners from one course 
were asked to do a test to determine their individual learning style prior to starting the 
course. On starting the course, they found the content was the same for all students, 
containing only written content. The test was perceived as adding no added value and 
learners reported feeling misled from the interactivity at the start. 


“We didn't just throw PowerPoints or PDFs online, or scan paper 
content. You must take the creative process of making content very 
seriously." Lead, Wave 2 


“Because it was the pilot one, they hadn't ironed out all of the issues 
with the online system and answering questions.” Learner, Wave 2 
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> For the providers, an important determinant of the quality of online content was 
the expertise of the developer. Content was a higher standard where a specialist 
had designed it, whether that be an in-house developer, or an external provider brought 
in for the role. High quality was achieved when the developer had the time to collaborate 
with relevant parties such as subject experts or awarding bodies. This resulted in well- 
rounded and well-produced materials. 


The quality of supporting materials suffered where there was less time or resource. 
Where online resources and platforms were simpler, tutors built content themselves. The 
aim was to reduce the time and cost of working with external partners. Achieving content 
of a professional quality was more difficult in these circumstances, as tutors’ expertise 
tended to lie outside of developing online materials. 


Some providers who had not previously developed online platforms had intended to use 
an external partner and had underestimated the time it would take to create, test and 
change content. In these cases, online platforms had not been developed in time, 
meaning learners had to work from paper workbooks instead. The materials lacked the 
anticipated added value of interactivity and usability which could have been gained from 
an online platform. Due to these barriers, providers recognised that they would produce 
better received materials if they could use an existing online platform or dedicate further 
resource and time to adapting existing content. 


7.3 Quality of teaching 


The qualitative interviews showed that many learners had little or no experience of adult 
education or lacked confidence in their ability to learn due to negative experiences in the 
past. Consequently, quality of teaching was paramount to maintaining their engagement. 
Feedback from both the qualitative interviews and the learner survey tended to be 
positive, although there were some areas in which improvements were identified. Three 
key factors were identified as important in determining the quality of teaching: 


e Personalisation of support; 
e Responsive communication; and 
e Consistency. 
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Personalisation of support 
Satisfaction with support and advice 


The learner survey results showed that there were high levels of satisfaction with the 
support and advice that learners surveyed received from staff. A large majority were able 
to contact staff when they needed to (87%) and received sufficient advice and guidance 
in relation to their course (85%). A slightly smaller proportion (80%) said that they were 
able to access technical support when they needed to. (Figure 27) 


Satisfaction with advice and guidance correlated with the number of communication 
channels that learners were able to use to contact tutors. Almost all learners (96%) who 
were able to contact their tutor in person, via telephone, via email or via online chat 
agreed that they were able to contact staff when they needed to. This fell to 88% of those 
who were able to contact tutors using three of these methods and 84% of those who 
were able to use two. Even among those who were unable to contact a tutor directly or 
who only had one communication channel available, 71% of learners agreed that that 
they were able to contact a tutor when they needed to. 


This suggests that making a range of communication channels available to learners may 
help them to engage with tutors. 


Figure 27 Learner views of staff support (survey data) 


To what extent do you agree or disagree with the following statements regarding 
support on your course? 


= Strongly agree m Agree m Neither agree no disagree u Disagree u Strongly disagree 
| have received sufficient 
advice and guidance in 36% 49% 6%) 7% 2% 
relation to my course 
| have been able to 
contact staff when | 34% 53% 6% 6% 2% 
needed to 
| have been able to | 
access technical support 29% 51% 9% 10% 19 


when | needed to 


Base: All learners surveyed (737); All learners whose course was partly or completely online (705) 


Consistent with the finding that learners surveyed were more satisfied with the 
== course if they had easier access to tutors, learners in the qualitative interviews 
highlighted that having personalised support was important to them. Where they 
found specific skills or subject areas more difficult, they reported that personalised 
support was important in meeting their practical needs. This type of support included 
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providing both positive and constructive feedback, and regularly checking in with learners 
to ensure they were on track. 


Learners explained that personalised support made them feel valued and engaged with 
the course. Best practice was found where tutors provided encouragement and moral 
support to those who were less familiar or comfortable with education, sometimes in a 
one-to-one scenario. 


“| needed to do things in my own time, but also have the one on one. 
| like that personal aspect of having someone to ask.” Learner, Wave 
2 


Learners also recognised that contact with fellow students gave them additional support. 
Classroom based learners across different courses indicated that a bigger class size was 
more beneficial than a smaller one. It was perceived to reduce feelings of isolation, 
enable learners to bounce ideas off each other and form networks lasting beyond the 
duration of the course. There was potential for this environment to be replicated online 
via features such as forums where learners could introduce and communicate with each 
other. Participation on these forums should be optional so as not to add an unnecessary 
layer of burden for learners, and should be monitored by tutors to ensure correct advice 
was being given. 


“| think the group helped massively with that. We were all having 
similar conversations about confidence and the course itself, so 
knowing that it wasn't just me really helped. Everyone was really 
supportive of each other.” Follow up learner, Wave 2 


Providers tended to deliver the required level of support where they had 

adequate resources to do so. Those delivering blended learning courses found 

that this was more costly in staff hours compared to traditional courses. It required 
face-to-face teaching as well as online monitoring by tutors and had therefore instead 
added an extra dimension to their role and increased workloads. This presented an 
additional challenge should flexible courses be scaled up. Tutors on these courses 
suggested a number of ways in which this could be managed. Firstly, they suggested that 
they would need additional time dedicated to supporting learners, and therefore a 
decrease in allocated teaching time. Secondly, other tutors called for more tutors to be 
involved in flexible learning. Lead providers, however, more commonly discussed setting 
clearer boundaries around the time that tutors should spend offering additional support. 


‘ Tutor buy-in to the concept of flexible learning was also considered important. 
Where tutors were engaged and passionate about flexible learning, they were 
often more willing to take on the extra workload. Some providers found this 


91 


difficult to achieve as their tutors did not want to change their way of working, often when 
they had been working at the same place for a long time and were not bought into the 
benefits of delivering courses flexibly hey emphasised the need for senior management 
to be enthusiastic about flexible learning, in order to facilitate a top-down attitudinal 
change, and for further support to be provided to tutors. 


Responsive communication 


The quality of teaching was also defined by the time that tutors took to respond to 
“= queries. Learners expected tutors to get back to them promptly, to avoid the 
need for them to chase a response and feel like they were burdening the tutor. 


“| felt that whenever you did contact them, it never seemed a burden 
to them. It was always marked on time and feedback given on time.” 
Learner, Wave 2 


“If | had a question, she would almost instantly email me back.” 
Learner, Wave 2 


The learner survey showed that the presence of a supportive tutor was the most common 
reason the course exceeded learners’ expectations. Conversely, a quarter of learners 
who were disappointed said that the main reason the course did not meet their 
expectations was due to poor support from the tutor. 


eal Similarly, among the learners who participated in the qualitative interviews, 
ir —, having responsive tutors allowed learners to complete their courses without 
delay and adapt their studies to changing circumstances, if a deadline 
extension was needed for example. As with personalised support, tutors who were 
responsive in communication tended to be those who had the necessary skills and were 
enthusiastic about flexible learning. However, due to the extra cost in staff hours and in 
staff training, providers were concerned about the extent to which this was scalable. 


Tutors met learners’ needs when communication was consistent and reliable. For 
example, learners referred positively to the availability of support outside of scheduled 
classroom hours and to drop-in centres arranged on provider premises or in community 
locations, where learners could come to ask any questions. Several providers 
implemented a system whereby learners could contact their tutor anytime through the 
platform and get direct feedback, as well as access optional face-to-face support 
sessions. On a course specifically for certain Muslim women who wanted to study 
exclusively from home, online access to tutors was available from 08:00 to 22:00, five 
days a week. 
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Learners felt dissatisfied where they did not hear back from tutors in a timely manner, or 
at all. These cases were often where providers had not implemented a system enabling 
communication between learner and tutor, such as through an online platform, and 
instead were reliant on tutors to communicate independently. 


Consistency 


Consistency was also important in determining the extent to which learners 
i thought teaching was of a high quality. Where learners had been promised 
something by a provider, they expected this to be delivered. This was important because 
they often chose to enrol on courses based on certain aspects that were appealing to 
them, and they expected to receive course certificates or other promised outcomes ina 
timely manner so that they could evidence their work. Where these aspects were 
delivered learners felt satisfied because the courses had helped them to achieve their 
aims. 

Providers maintained their commitments where they had adequate resources 

and time to plan and work with partners, as well as enough staff to cover for last 
minute changes. On the other hand, some providers lacked consistency because of time 
constraints, meaning less time to collaborate with partners. For example, one provider 
had promised that a course would culminate in a meeting with employers, but this did not 
happen. For some learners this aspect had been an important consideration in deciding 
to enrol on the course, so the lack of consistency was disappointing. 


In other cases, providers did not acknowledge the impact that breaking commitments to 
learners would have, such as in the case of one provider who had promised an 
introductory face-to-face session which did not then go ahead. This led to dissatisfaction 
with the course as well as with the provider as a whole. 


7.4 Course accessibility 


The survey showed that learners consistently highlighted other priorities in their lives, 
such as caring responsibilities or employment, and thus the courses needed to be 
accessible. The most common reason given for choosing a flexible course was to fit 
study around existing commitments. 


In determining how accessible courses were to learners, there were four important 
factors: 


e Timings; 

e Locations; 

e Workload; and 

e User-friendly content. 
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Timings 


There were a variety of ways in which courses were flexible in their timings. Some 
“*" courses with face-to-face contact tried to make timings flexible to fit around the 
target audience by, for example, having it outside of regular working hours or holding 
sessions once a week. Where the course was held predominantly online, /Jearners were 
generally able to complete it at a time that suited them, giving them complete flexibility. 
These flexible timings enabled learners with competing priorities or who were working 
shift work to participate. 


“| could do it completely at my own pace, which | found a lot better 
than anything else | could do.” Learner, Wave 2 


Learners were positive about the accessibility of timings where the provider had been 
responsive to their needs. In these cases, providers had an existing knowledge of learner 
needs through their own experiences or partner knowledge or had adapted how and 
when the courses were run based on learner feedback. 


v7 ] Some providers delivered flexible timings by giving tutors more autonomy in 
Fe organising when to meet with learners. This gave learners more input into 

choosing times that suited them as they often had a closer relationship with the 
tutor than those designing the course. However, such a solution presented issues with 
scalability because it placed a large burden on the individual tutor. As learners all had 
different preferences, it led to a system of one-to-one tuition, which was time and 
resource intensive for tutors. This was highlighted as something that would need to be 
taken into consideration going forward, with providers currently uncertain of how to 
balance a flexible, personalised service with sustainable workload for tutors. 


Some providers found that taking an approach whereby there was no defined 
— start and completion date, whilst highly flexible, was problematic because it meant 
that learners took too long to complete the course. Whilst they acknowledged the 
importance of learners being able to study in their own time, they said that in the future 
they would market the courses with tighter timelines in order to prevent learners from 
taking too long. Providers suggested that they would need a trial and error approach to 
ensuring learners had flexibility around when to complete the course while still 
maintaining learner motivation. 


Locations 


Ss Courses were carried out in a variety of locations in order to be accessible for 
“* learners. These included the providers’ own premises, community centres, public 
spaces such as cafes, or wherever the learner was (for online content). 
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When content was available online, learners could access the content as and when they 
liked to fit around their lives. This was a broadly successful way of achieving accessibility. 
An example is a course where learners could continue with their learning units on iPads. 
However, this approach was seen as costly, and in order for such content to be 
experienced as accessible, learners needed personalised support and responsive tutors. 


Delivering education in community centres was especially useful for providers 
— with many learners based in rural areas, as these centres were often in close 
proximity to their homes. Where providers enabled meetings to take place in public 
spaces this was convenient for learners as they could choose where suited them. 
However, it required tutors to be very flexible and increased staff hours due to extra 
travel time. 


“It was really easy to get to. | used to drop the kids off and then go.” 
Learner, Wave 2 


"At my own leisure, at home. Spot on." Learner, Wave 2 


There were a minority of learners who did not have access to the internet at home. Some 
providers made computer rooms available for learners to use, but this did reduce 
flexibility as it was dependent on the provider’s opening hours. 


Workload 


The learner survey showed that most FLF learners (60%) spent less than 10 hours per 
week studying for their courses. Some courses seem to have required a heavier 
workload, with 12% of learners surveyed saying that on average they spent more than 20 
hours per week studying for their course. (Figure 28) 
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Figure 28 Hours spent studying per week (survey data) 


On average, how many hours a week do/did you spend studying for your course? 


Base: All learners surveyed (737) 


Perhaps reflecting the relatively low average number of study hours required by most 
FLF courses, relatively few learners reported that they struggled to fit studying for their 
course around their other responsibilities. 


Among those who did not work and did not report any caring responsibilities, only 16% 
said they found it fairly or very difficult to study. This rose to 27% of those who work and 
28% of those with caring responsibilities. Around half of learners who were employed 
(5%) or learners with caring responsibilities (49%) said they found it very or fairly easy to 
fit studying around their other commitments. A larger proportion of learners without caring 
or work responsibilities (67%) said they found it easy to fit studying into their lives. 
(Figure 29) 


96 


Figure 29 Ease of fitting study around other commitments (survey data) 


How easy or difficult do you find it to fit studying for your course around your...? 


u Very easy mu Fairly easy = Neither easy nor difficult mu Fairly difficult a Very difficult 


Your life 29% 38% 17% | 11% 5% 


Work Schedule 17% 34% 22% 17% 10% 


Caring 


0, 0, 0, 0, [ey 
Responsibilities ae 29% 22% 20% 9% 


Base: All learners surveyed who are not currently in work and don’t have any caring responsibilities (144); 
All learners surveyed who are currently in work (466); All learners surveyed who have caring 
responsibilities (406). 


Note: Some learners had both work and caring responsibilities. 


The qualitative interviews showed that learners expected to be able to fit their 

work around other commitments, and to spend no more time than the provider 

had advertised. Where providers were able to successfully deliver on expectations 
around workload, they had allocated sufficient time to develop and test the content. 
Where workload was higher than anticipated, it became even more important for tutors to 
provide additional support and be responsive to learners’ questions. As previously 
mentioned, where providers could adapt content from existing courses or already had an 
online platform available which they could work from, this was helpful as it made the 
process less time and resource intensive, freeing up time to provide extra support. 


The workload was manageable, and we were told we could take time 
out of our other work, so that wasn't a problem — Follow up learner, 
Wave 2 


User-friendly content 


For learners to engage with materials, they needed to be user-friendly. For 
7 workbooks and online materials this meant content that was easy to follow and 
navigate. This was especially important for learners who lacked digital skills, who often 
needed a hands-on tutorial at the start of the course to be able to use materials 
confidently. 


“| needed support at the start. | went in to try and get a hand to find 
out what to actually do. | was able to do it from home once | knew 
how it worked.” Learner, Wave 2 


“The online elements were more cumbersome because it wasn’t that 
easy to use. Writing on paper was easier for me personally.” Learner, 
Wave 2 


Providers who achieved this standard were those who had spent sufficient time 
= on testing and reworking the materials according to learner and tutor feedback. 
One provider emphasised the importance of tutor engagement in helping to facilitate 
online learning, as they found that where tutors had provided hands-on support, learners 
described finding the system easy to navigate. 


However, despite online platforms being designed to be user-friendly, some providers 
found them unsuitable for certain cohorts. For example, one provider developed learning 
for ESOL learners but found that their level of English simply was not high enough to be 
able to confidently use the system. 


Case study 6. Delivering accessible learning 


- . Online learning for courses in Functional skills, Maths and 
Open University English (level 1 and level 2). 


Open University had previous experience of delivering flexible leaming. For the new courses, they worked with 
three partner colleges who helped to develop the course content. 


As all courses were delivered via distance learning, learners 
could study at times that suited them. 


Flexible timings 


Learners could study from any location as learning was online. 


Flexible locations They could take their exams at any of the three partner colleges. 


Tried and tested A tried and tested platform ensured the content was easy to use 
content and suitable for distance leaming. 
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8. Course outcomes 


This section provides a brief overview of learners’ perceptions of their courses, 
continuing with an in-depth discussion of learner’s objectives, and which factors helped 
them to meet their goals. 


Golden Rules for Course Outcomes 


For those looking to build soft skills, having something physical or tangible to take 
away (for example, a certificate or text to add to their CV) increased the value they 
placed on the course and subsequently boosted their confidence. 


Learners highlighted that providers missed opportunities to further their engagement 
with education. Learners discussed how providers were rarely proactive in following up 
with them at the end of the course. They suggested that being given tangible options 


for next step courses may have motivated them to continue building on their skills. 


Moreover, learners also highlighted wider contextual barriers to continuing their 
education. More advanced courses came with both logistical (for example, additional 
time) and financial barriers. 


Learners who had experienced positive outcomes commonly recommended flexible 
courses to friends and family, demonstrating that word of mouth could be an effective 
tool for future recruitment. 


8.1 Learner views of course impact 


Learners had positive views of the impacts of FLF courses on their own skills and 
abilities. Most survey respondents agreed that they had learnt new skills on the course 
(87%). This varied by age with almost all (92%) of 16-34-year olds saying they had learnt 
new skills in comparison to 79% of those aged 50+. (Figure 30) 


Most learners agreed that they had gained confidence (80%) on their course. Carers 
were more likely than learners without caring responsibilities to say that their course had 
improved their confidence (84% compared with 76%). Again, learners aged 16-34 (86%) 
were more likely than learners aged 50+ (73%) to say that they had gained confidence. 


Learners who were quantitively surveyed were optimistic that they had benefitted from 
the courses and were generally confident that these new skills would have a positive 
impact on their future careers. Those not in paid work when surveyed were asked if they 
thought their chances of finding paid work in future had improved as a result of doing 
their course, to which 73% agreed. Sixty-three per cent of those in work when they took 
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part in the survey agreed that their chances of earning a higher wage in future had 
improved. 


Again, a higher proportion of younger learners aged 16-34 agreed that their chances of 
earning a higher wage had improved when compared to those aged over 35-49 or 50+ 
(83% compared to 56% and 45%). 


Figure 30 Perceived impacts of CLP courses (survey data) 


To what extent do you agree or disagree with the following statements? 


= Strongly agree m= Agree = Neither agree no disagree = Disagree @ Strrongly disagree 


| have learnt new skills 41% 46% a yal, 
iio 35% 45% 13% 6% 19 


My chances of finding 

paid work in the future 28% 45% 16% 8% 3% 
have improved 

My chances of earning 


a higher wage in 23% 40% 21% 11% 5% 
future have improved 


Base: All learners surveyed (737); All learners surveyed who were not in work at the time of the interview 
(271); All learners surveyed who were in work at the time of the interview (466) 


8.2 Intended outcomes 


Learners signed up to flexible courses with different intended outcomes in mind. Based 
on the qualitative interview data, three groups of learners were identified, with each 
having different characteristics, levels of forward planning and factors influencing whether 
they achieved their outcomes. The three groups included learners aiming to: 


e Build soft skills such as confidence or interpersonal skills; 

e Improve a specific skill or gain a specific qualification related to their professional life 

e Achieve a longer-term aim of which the course was just one element (for example, to 
gain the relevant qualifications to be able to complete a degree). 


For each group, there were a range of factors that influenced the extent to which learners 
were able to achieve their intended outcomes. There were also steps that could be taken 
to encourage learners to further their educational journey. 
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8.3 Building soft skills 


The survey results showed that a larger proportion of learners studying on courses taught 
partly online or entirely in person agreed that they had gained confidence than learners 
studying on courses which were taught entirely online (83% and 84% compared with 
60%). 


These findings were supported by the qualitative data which showed that some 
interviewees enrolling onto flexible courses sought to build soft skills such as improving 
confidence or interpersonal skills. Rather than seeking a tangible outcome such as a 
specific qualification or skill to be applied in a work environment, they aimed to build skills 
that would be useful in their personal lives. For example, some focused on improving 
their confidence in interacting with people not in their immediate circle of contacts, while 
others were looking to find a hobby. Learners that fell into this group tended to: 


e Have minimal or no experience of adult education: some learners in this group 
were new to adult education and expressed that they were uncertain of what to 
expect before they started. They tended not to have clear expectations of what the 
outcomes would be but aspired to build confidence both within and outside of an 
education setting. 

e Have had negative experiences of education during school: some learners in this 
group reported difficulties in education in their past, such as finding the academic 
or social side of school difficult. These learners sought to build their confidence in 
education, rather than seeking a specific qualification. 

e Not currently be in paid employment: learners who were not in paid employment 
had less certainty over what their goals should be. They therefore sought to build 
on softer skills that would be relevant to their everyday lives. Some of these 
learners had been referred to the courses by the JobCentre Plus. 


Learners looking to build their soft skills tended to enrol on courses that were more 
general, for example foundational or employability courses, or they signed up to courses 
aimed at those new to the education system, such as English and Maths Level 1. 


Generally, these learners found that they had achieved their intended outcomes. 
However, some areas for improvement were identified. Three key factors influenced the 
extent to which the intended outcomes were met: 


e The level of access to support throughout the course lifecycle; 
e The consistency of the course with the specification outlined at the start; 
e The extent to which learners were given a tangible takeaway from the course. 


Access to support throughout the course lifecycle was essential for this group of learners 
to achieve their intended outcomes, particularly improved confidence. This group started 
off the process with low confidence, commonly needing encouragement from providers to 
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sign up and start the course. Learners reported a greater level of confidence starting the 
course where they had received either a face-to-face introductory session or ongoing 
personalised communications online. 


“Now | feel more comfortable to start conversation. Before | was 
shyer to speak because | was afraid that | would say something 
wrong.” Learner, Phase 3 


Continued support was also necessary once the course had begun. Learners highlighted 
that they felt more motivated when tutors proactively nudged them to continue with the 
course. This demonstrated that they were valued by the tutors and increased learner 
confidence to reach out to tutors where they needed further support. Learners highlighted 
that support did not need to be face-to-face, but that there was more satisfaction where 
this was an option. 


This group of learners rarely had tangible plans in place for how they would apply their 
skills after the course was complete. They therefore required additional support in moving 
them onto the next stage in the journey, where their learning would have a more specific 
aim related to employment. Where learners felt their intended outcomes had been met, 
they had a greater level of confidence to enrol in further courses. However, they also 
relied on tutors encouraging them to pursue further education and informing them of 
relevant courses that they could take. This built their confidence to feel that they would 
have the ability to complete another course and that it would be suitable to their needs. 


Finally, learners highlighted how having something physical or tangible to take away from 
the course boosted their confidence more than expected. While this group were not 
generally taking part in the course for this purpose, having something tangible to show for 
their time (for example, a certificate or text to add to their CV) increased the value they 
placed on the course and subsequently boosted their confidence. Learners highlighted 
this as an outcome that was more likely to give them the confidence and motivation to 
pursue further learning related to a specific skill or qualification. 


“| think | would’ve been more motivated if it wasn’t just the JobCentre 
saying, ‘do a course to get a job’. Knowing exactly what | would have 
got out of it might have pushed me more.” Learner, Phase 2 


8.4 Gaining a specific skill or qualification 


The next group of learners enrolled on flexible learning courses with the aim of achieving 
a specific qualification or improving a specific skill that could be applied directly to their 
professional lives. These learners tended to already be in paid employment. The courses 
they were enrolled onto therefore had a greater level of focus than for those looking to 


102 


build soft skills, and included courses such as Enterprise skills, Management skills and 
Social Media Marketing. 


The trigger to signing up for the flexible courses varied. Some were self-motivated, 
having identified a skills gap for progression in their current or desired career. This group 
were generally confident in their ability to navigate flexible courses, but less confident in 
the specific skill they were looking to improve. They had high levels of motivation and 
engagement with the course. Others had been asked or mandated to complete a course 
by their employer. The level of confidence in the specific skill they had been asked to 
improve on varied. Those who already had high levels of confidence tended to be less 
motivated as they struggled to see the purpose of completing the course. 


In general, learners who participated in the qualitative interviews also reported that they 
had achieved their intended outcomes, however some areas for improvement were 
identified. Three key factors influenced the extent to which the intended outcomes were 
met: 


e The level of support received from employers and colleagues; 

e The level of support received outside of work; 

e The extent to which the course aligned with the reality of their working 
environment. 


The level of support that learners received from employers and colleagues influenced the 
extent to which learners felt they had achieved their intended outcomes. Those that had 
been asked by employers to complete the course generally had high levels of support. 
Shift times were adapted to suit the course times, or the learners needs, and some were 
permitted to complete the course during working hours. However, some learners 
reflected that employers had not accounted for adequate time for additional course work. 
This left them struggling to fit course commitments around other responsibilities in their 
personal lives. 


Support for learners that were self-motivated to sign up for the course varied, based on 
how much value the employer placed on the course. Where the course was perceived to 
be of benefit for them, the employer regularly provided additional help on assignments 
and allowed the learner to move or swap shifts were needed. However, learners were 
commonly also reliant on colleagues to pick up shifts when the course workload was 
greater than expected. Where learners were not offered support from employers and 
colleagues, completion was perceived to be more difficult despite the flexibility of the 
courses. 


Learners also relied on support outside of the work environment in order to achieve their 
intended outcomes. They commonly had commitments outside of their employment, 
largely involving caring responsibilities. Learners commonly mentioned that their courses 
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took up more time than anticipated, and so having the support of friends or family with 
other responsibilities was important in ensuring they could dedicate enough time to the 
course and achieve their intended outcomes. Where this did not happen, learners found 
it difficult to keep up with the course and started to get feelings of disappointment and 
reduced motivation. 


As discussed, this group of learners signed up to their course to achieve a skill 
specifically related to a working environment. Their motivation, and therefore outcomes, 
were greater where learners were could see how they could apply the new knowledge in 
their given workplace. For those who were self-motivated to sign up to the course, this 
facilitated continued engagement and motivation. For those who had been asked or 
mandated by an employer to do the course, this motivated them to put in more effort and 
in some cases find unexpected enjoyment or benefit from the course. 


The research identified some missed opportunities in keeping this group of learners 
engaged with education. Learners discussed how providers were rarely proactive in 
following up with them at the end of the course. They suggested that being given tangible 
options for next step courses may have motivated them to continue building on their 
skills. 


Moreover, learners also highlighted wider contextual barriers to continuing their 
education. More advanced courses came with both logistical and financial barriers. 
Firstly, these courses were perceived to take up more time and were not commonly 
flexible in their delivery. The financial cost of continuing their education was also 
considered too high to be able to commit. 


[When] Working it can get quite complicated to do a course ata 
place, online is much better. Learner — Phase 3 


I've got the certificate, | can carry on with my life and not worry about 
it anymore — Learner, Phase 3 


| don't feel so worried and scared when | write my emails. | feel so 
much more confident, definitely. - Learner, Phase 3 


8.5 Pursuing longer-term outcomes 


The final group of learners took part in flexible courses with the aim of achieving a longer- 
term goal. Instead of the outcome of the course being their focus, these learners used 
the course as a stepping-stone or as one element of a longer-term aim. For example, 
learners in this group were using the courses as a way to work up to a degree, move 
sectors or start their own business. 
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Learners aiming for longer term outcomes generally had existing soft skills that could be 
applied to the course. For example, they tended to have a high level of confidence in 
their ability to complete the course and were assured in their interpersonal skills. They 
were commonly highly motivated to complete the course and pursue the relevant next 
steps and had spent a significant amount of time researching both the immediate course 
and what they would need to do afterwards to achieve their overall aim. 


This group of learners were enrolled on a varied set of courses. Learners that were in the 
earlier stages of long-term plans more commonly enrolled on Maths and English courses 
at various levels. They recognised that these were a minimum application requirement for 
numerous pathways. Others that were further along in their plans tended to enrol on 
courses that were geared towards specific sectors or skills. These included courses 
teaching skills for self-employment, Cognitive Behavioural Therapy and Beauty Therapy. 


| want to be a paramedic, and to do that you also need maths and 
English GCSE. — Learner, Phase 1 


Learners in this group commonly discussed the outcomes of the course in relation to their 
longer-term aims. They were largely positive about the role of the flexible course in 
accelerating them towards their end goal. 


It's made me continue education, and to want to learn more. It's 
helped towards the end goal for me. — Learner, Phase 1 


They were highly self-motivated, meaning that providers did not need to offer further 
support or encouragement to them. Learners were most positive about the outcomes 
where the course fit the description provided at the outset, although were still likely to 
complete the course if there were some aspects of the process that they did not like or 
perceive to be useful. It did not tend to distract them from their objectives because they 
were completing it in order to be able to begin the next step. 


This group were more likely to have chosen the course specifically because of its flexible 
delivery format. In general, it was very important for them to be able to balance their 
studies with their other time commitments in order for them to complete the course, and 
therefore continue the path to their desired long-term destination. Flexibility in terms of 
location, time and pace were all relevant for this group but the type and degree of 
flexibility depended on the circumstances of the individual learner. 


I've been wanting to do this nursing apprenticeship for a really long 
time, and it's enabled me to do that without giving up my working 
hours. — Phase 1 
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8.6 Unintended outcomes 


As well as learners achieving some of the outcomes they expected as outlined above, 
learners also noted some outcomes that they had not anticipated. 


The unexpected outcomes could generally be divided into two categories: 


e Professional outcomes, i.e. relating to learners’ current and future work prospects 
e Social outcomes, i.e. pertaining to learners’ personal and social lives. 


8.7 Professional outcomes 


While learners commonly registered for courses with the anticipation of gaining some 
professional skills, they also gained some additional and unexpected skills that could be 
applied in the workplace. Learners on maths and IT courses referred to skills that helped 
them work more efficiently, for example learning shortcuts that they could use when 
working with spreadsheets. Similarly, some learners who were aiming to become self- 
employed learnt skills that they had previously thought they would need to employ 
someone to do for them. This included calculating taxes which in the longer term offset 
the anticipated costs of accountants. 


However, unintended professional outcomes were not always positive. While gaining a 
qualification was largely viewed as invaluable for career progression or finding a new job, 
there were some concerns raised that the courses had not adequately prepared learners 
for their desired roles. Learners who found themselves in new jobs sometimes felt 
unprepared for the demands of the job. They made suggestions for how courses could 
have been improved — for example, by offering contact with employers. Despite this, it did 
not make them less likely to pursue education in the future, whether with that provider or 
otherwise, due to the value of gaining a job in their desired field. 


Confidence | am lacking a lot, because they didn't teach you what it's 
like in the outside world. — Learner, Phase 2 


On the other hand, there were cases where learners’ experiences were negative in a way 
that could dissuade them from enrolling on further courses. This was an issue especially 
in cases where providers had not delivered essential aspects of their course as 
promised. An example of this is where a provider promised a networking opportunity with 
employers that did not materialise, resulting in huge disappointment for learners for 
whom such an opportunity had been a primary incentive for completing the course. The 
experience made them sceptical of the provider's motives, and ultimately made them less 
likely to pursue further education with that provider. 
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"It was devastating. After three weeks of my time, and the sacrifices 
my family made." - Learner, Phase 2 


8.8 Social outcomes 


Learners generally expected to improve their confidence, but beyond that had rarely 
considered that there would be social outcomes of taking part in their courses. 


Learners who had more classroom time commonly discussed the unintended benefits of 
having contact with their peers. They were surprised at the level of support and social 
connections made via their courses as they went through the experience together. For 
learners who were feeling socially isolated prior to studying, this was considered to be 
particularly beneficial as it helped to alleviate their isolation, and in turn improve their 
well-being. For other learners, connecting with fellow students helped them feel 
supported and raised their confidence in being able to complete the course successfully. 


Learners that were enrolled on online only, or predominantly online courses, did not 
discuss the social connections made through their courses. This indicates that meeting in 
person was important for learners to make meaningful connections with their peers. 
However, this was not considered to be problematic because they did not go in with the 
intention or expectation of forming new social connections. 


Learners who had had a positive experience of a course sometimes recommended the 
provider or course to people close to them, such as family members or neighbours. As a 
result, some individuals that they referred had since enrolled on a course with the aim of 
improving their job prospects. This demonstrates the importance of running successful 
courses as it in turn leads to further enrolments — something that we have already 
highlighted in chapter 6 as being important for course recruitment. 


I've got three daughters and two of them are doing courses in maths 
and English and that. — Learner, Phase 1 


While the courses were generally found to have positive impacts on learners, there were 
some short term adverse social effects raised where courses did not meet expectations. 
Learners highlighted that courses were sometimes more time intensive than they initially 
expected. As a result, some learners expressed having less time available for important 
aspects of their personal lives, such as hobbies or spending time with family. Learners 
tended to see this as an inevitable consequence in the short-term and did not consider it 
a deterrent for future education. Learners felt that if the workload was manageable and 
deadlines were flexible, it would have helped them to avoid missing out on other activities 
they wanted to do. 
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| suddenly found | had lost seven hours so there were home, family 
things | wasn't doing during that time. — Learner, Phase 3 
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9. Monitoring mechanisms 


Providers took a variety of approaches to monitor the progress and success of their 
courses. This section explores the different methods used and the extent to which these 
gave them relevant information that could be used to improve course delivery. 


Golden Rules for Monitoring Mechanisms 


e The process of monitoring learners’ progress is time consuming. Providers need to 
ensure they are allowing adequate time for tutors to manage the process and are 
distributing the workload to reduce the burden while ensuring learners are receiving 
the encouragement needed. 


Tutors should be upskilled on how to interpret monitoring data to ensure they feel 
equipped to fulfil their responsibilities. 

Providers should gather feedback as the course progresses, as opposed to solely at 
the end of the course. This would increase the opportunity and motivation for learners 
to complete the feedback and give providers the opportunity to act on feedback and 
rectify any issues identified. 


9.1 Monitoring online progress 


Providers that used online platforms as part of their flexible offer were able to 

track online learner progress. Generally, this came in the format of a data file 
outlining the stage of the course that each learner had reached, as well as outcomes of 
assessments. Some providers mentioned having access to more detailed data, including 
the amount of time that each learner had spent on each page or topic. 


The rationale for using this data was largely consistent across providers. They welcomed 
the ability to identify key areas that were causing learners difficulty, both at a group and 
individual level. This enabled them to offer encouragement to individual learners who 
were progressing slowly and target broader support where necessary. 


“We had analytics on individuals and different groups including how 
often and when they accessed the platform, for how long and how 
long of that was active. We weren't expecting such low 
engagement...that was the first surprise.” Non-case study lead 


Providers varied regarding who had responsibility for tracking the progress data and 
taking the associated actions. Where there were fewer learners signed up, some 
providers allocated the responsibility to one or two members of staff. They reviewed the 
data and followed up with learners where progress was slow. These individuals tended to 
be in administrative roles rather than being directly involved in course delivery. 
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Monitoring the data in this way held some benefits. First, the responsible employees 
were often used to interacting with similar data and so became familiar with how to 
interpret it quickly. Second, having a small number of people responsible ensured a good 
level of oversight and consistency in their approach. Due to this, these individuals were 
able to report back on ‘big picture’ outcomes as well as on individual level concerns. 


However, there were concerns raised about the sustainability and scalability of this 
approach as providers were commonly aspiring to achieve a higher volume of learners in 
the future. They suggested that the individuals currently carrying out this process would 
face difficulties with a higher number of learners and so the quality of the actions they 
took would be reduced. They recognised a need to assess alternative ways to monitor 
progress going forward, but at the point of interview did not have alternative measures in 
place. 


“Some of our partners would need more staff to monitor learner 
progress if scaled up to make sure help would continue to be offered 
to those with slower progress.” Lead, Wave 1 


7] In contrast to this centralised approach, it was more common for providers to ask 
‘4 ry each tutor to track the progress of students within their cohort. This approach 

overcame some of the issues faced by taking a more centralised approach. The 
workload tended to be more evenly spread across the organisation, reducing the amount 
of pressure placed on one individual. While there were fewer concerns raised about how 
this process would be managed should the volume of learners increase, some tutors 
were concerned about the rise in workload should they have to monitor a greater number 
of learners. They emphasised that the workload would have to be spread across a 
greater number of tutors, and therefore more tutors would have to be upskilled. 


Having oversight of up-to-date learner progress also enabled tutors on these courses to 
more effectively target their support. They were able to intervene with individuals where 
necessary, and in a more personal and supportive way. They were also able to better 
identify where larger portions of the group were struggling on a specific element of the 
course. 


However, providers using this method of monitoring progress suggested that some tutors 
were concerned about having to interpret data. In some cases, this led to specific tutors 
who were more comfortable working with data being used to deliver flexible courses. 
They were concerned about the ability of other tutors to use the data in the same way 
should the flexible offer be upscaled. 
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“There are some of them [tutors] who have never had to use 
spreadsheets like we're giving them. They don’t know where to start 
and to be honest, don’t want to.” Non-case study lead 


Overall, irrespective of the method of monitoring online progress, the process was found 
to be more time intensive than anticipated. This resulted in those responsible feeling a 
burden or needing additional resource to meet the needs of the course. 


9.2 Quantitative surveys 


, Providers interviewed at a later stage in their journey (wave 2 case studies and 
non-case studies) more commonly discussed carrying out quantitative surveys to 
monitor course success. 


Quantitative surveys, run separately by providers themselves, were conducted with 
learners enrolled on flexible courses. A small number of pilot providers also included 
tutors and mentors in the data collection. They were generally carried out once the 
course was complete and aimed to identify areas of success and areas where 
improvements could be made. 


Across providers, the volume of survey responses was lower than anticipated. While 
some providers attributed this solely to having fewer learners enrolled on the course than 
expected, a number of learners acknowledged that they had forgotten to return their 
survey because it was issued at the end of the course. The low volume of responses 
meant providers gained indicative insight into learner experience, rather than reliable 
conclusions that could be actioned. One provider was taking steps to increase response 
volume by moving the survey online. 


A small number of providers found their surveys to be a success. These tended to be 
providers with higher numbers of learners, and those with existing links with partners who 
had recruited on their behalf where the partner had played a key role in encouraging 
feedback. 


9.3 Qualitative feedback 


Providers with low numbers of enrolled learners more commonly gathered 
qualitative feedback from both learners and tutors. This came in a variety of 
formats, including focus groups, interviews, anecdotal evidence and video feedback. 


For some providers, qualitative feedback was used as a contingency plan when the 
volume of learners was too low to draw rigorous quantitative metrics. These providers 
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recognised the need for a greater depth of insight than the ILR data alone could provide 
them, with qualitative feedback seen as a way to interpret the ILR data. 


Other providers had always intended to use qualitative feedback because they 
recognised the pilot nature of their courses. They thought that at the early stages of 
development they needed to understand the detail of course elements that needed to be 
refined or improved. For example, one provider identified parts of a course that learners 
had consistently found difficult. They were in the process of trying to reduce the amount 
of content in the module and simplify what was there. 


“We're in the pilot stage at the moment. It’s all about building our 
knowledge of what is wanted and how we can improve.” Non-case 
study lead 


There were a small number of providers that were more selective in the materials that 
they gathered. Instead of using feedback for course improvements, they were looking to 
identify good news stories to use for promotional material. 


9.4 External partners 


lags Less commonly, providers commissioned external partners to undertake 

™ monitoring and evaluation work on their behalf. These providers recognised the 
benefits of building an in-depth understanding of how their courses were performing, and 
what could be done to improve outcomes. To reduce the burden on the internal team, 
they had opted to outsource this work. 


Delays were incurred at a variety of timepoints in the lifecycle of course development and 
delivery. These delays impacted the ability of partners to carry out the monitoring and 
evaluation activities that they had originally planned. The delays incurred during set up 
impacted on the recruitment of learners (see Section X). Having fewer learners than 
anticipated resulted in partners having to adapt their original plans. Moreover, partners 
were left with less time to deliver their work. 
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Case study 7. Monitoring and evaluating progress through a partner 


Workers Educational A virtual learning platform to supplement classroom based 
Se teaching in courses including Maths, English and English 
Association (WEA) for Speakers of Other Languages (ESOL). 


WEA worked with Behavioural Insights Team (BIT) to develop and test behavioural interventions aimed at 
encouraging continued course engagement and to track what students were doing next. 


BY=\V{-) (e) e110] BIT designed bespoke behavioural interventions to test including 
behavioural a goal setting activity and peer support, values affirmation and 

: ‘ peer support. They also helped WEA to track what students were 
interventions doing after completing the course. 


WEA and BIT found it challenging to get learners to return their 
consent form to allow for contact beyond the course ending. This 
limited the analysis they could do on learner’s next steps. 


Problematic 
engagement 


The given timescale gave BIT until March 2019 to be involved. At 
a Wfelalmn tiaat=ssyer=|(=4<) this point there wasn’t enough data or time to provide detailed 
analysis 
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Appendices 


Appendix 1 Trends in course uptake 


Appendix 1.1 Projects where uptake was higher than expected (based 
on uptake of the comparison courses) 
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Figure 31 London Skills and Development Network uptake 


c.160 additional learners 
vs. comparison courses 


198 FLF learners in 2019, 
2015 2016 2017 2018 2019 48% of all 2019 learners 


——FLF courses ===—Comparison courses 


Figure 32 Bristol Community Learning uptake 


Bristol Community Learning 


c.400 additional learners 

i eat vs. comparison courses 
525 FLF learners in 2019, 
2015 2016 2017 2018 2019 21% of all 2019 learners 


——CLP courses ===Comparison courses 
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Figure 33 Derby Business College uptake 
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Figure 34 Fareport Training Ltd. Uptake 
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Figure 35 NA College uptake 
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Figure 36 South Devon College uptake 
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Figure 37 The London College of Beauty Therapy uptake 
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Appendix 1.2 Projects where the number of FLF learners was at least 
as great as the estimated increase in uptake (based on uptake of the 
comparison courses) 


Figure 39 Dudley College of Technology uptake 
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Figure 40 Hounslow Adult and Community Learning uptake 
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Figure 41 National Numeracy uptake** 
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Figure 42 Open Award uptake 
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Figure 43 Skills and Learning Bournemouth, Dorset and Poole uptake 
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Figure 44 St Helen's Chamber uptake 
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Figure 45 Access Skills Ltd. Uptake 
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Figure 46 Durham County Council uptake 
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Figure 47 Inspire uptake 
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Figure 48 Weston College uptake 
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Appendix 1.3 Projects with little or no increase in uptake (based on 
uptake of the comparison courses) 


Figure 49 Kensington and Chelsea College uptake 
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Figure 50 Newham College uptake 
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Figure 51 Rutland Adult Learning and Skills Service uptake 
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Figure 52 The Open University uptake 
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Figure 53 Workers’ Educational Association uptake 
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